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(54) SYSTEM FOR KEEPING FORCIBLY AND SEQUENTIALLY ORDERED PACKET FLOW IN RING NETWORK 
SYSTEM PROVIDED WITH BUSY NODE AND DEFECTIVE NODE 

(5 7)Abstra ct: u > ^ w 

PROBLEM TO BE SOLVED: To provide a system for keeping a reliable 
packet distribution in a ring network provided with the support of a 
forcibly ordered non-idenpotent command. 

SOLUTION: Respective consumption side nodes in the network 

includes the final and previously known excellent packet and its order VT^m. / \ [ ~7^y^ 

number record and holds the order of the packets passing through the 
nodes and the record of respective packet states at the time point of 
passage. When an error condition inside affirmation response j 
concerning the packet is detected, a generation side node re-transmits 

the whole packets starting from the final and previously known "i 'V 

excellent packet. The respective consumption side nodes processes or 
rejects the re-transmitted packets including the packet with the 
possibility of being the processed one. But it is recognized by the 
packet and state record concerning the whole packets. 
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tcDtcTi^tl-TT'fo^o 112 3 4b^ii>*iii2 4{;i^i-^* 

[0 0 5 2] a^tC. 112 5i7:>CRC^;55cf^^{l. 2 
4. 40-Crno__xmitJ ibc^.b^*:y h^y^:^ h 

h(DXh^o ^v^^n 6 4i:be^^if7K— 

[0 0 5 3] ^2 1 (D^x:iy^m<om^ 
[3 2 7(1. {I18*5j:U^llj4 OOi:-?) Sim/— K (ill 
m/-K(ll]7) c7:)^^^/^:7^^p ^^'^^HT^fot?. sso 

[0 0 5 4] m2 7p^CDy<^y h(Dmt\^i:^m:^mc^T 
■(Dmy)Xh^o -y<^yhn. ^):y^\^vV2%, 9 0 
^^"bT^v^y T^-rif 2 6. 8 0 ^iloT^m K}::A 
5o CRClt^^ 8/ 1 07=^'::i- Kf^^2 6. 7 0(1^ 
^<'^y hf5-^&Sl'9l^#. ^^°^';/ h ^ 3 2 h r e 
c_p k t U>?>^^ 2 6. 6 0 (Ca— Ki-^o 
40 ~/<^y VCD^mt^^^-^fX.. riDl-v?;^^f^T^II^yN' 

y V\-t'iX'T(D\^(D^^tj, 

-^(r>y-Y\z.T Y\^:^n'&^Mz'&\%y<^yV (^* 

-fiilOy- KtCT KUx;fi^^tL/ci||^t/^^^:y h (^* 

-{&£Oy- K(CT Y\y^n'&^Mz^'&lt^'^y'^^y V 

e q u e s t_ i n^#•^b^t^'J (^*/^^>;/ V) %tz\'X r 
50 esponse_in ^^'^b^T^lj (^L^^^^^-^ :y h ) {Z'tfl 
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tl. a c k#-^tT?ij2 6. 1 5 Oi^##iA^iX^„ 
-Sffl/- K • y<^iy hitt e s t_p k t KtdS 
tiA^n^o ^<^y h(DmM^^ t e s t_p k t Uv^;;?. 
^F^(cfo5:^'^. p r o d u c e r I d^^trS 2 h'^y 
h:6^r e c_p k t Ui^y^ ^ [^{ZA^X HiJ^r^^ — 

y K^iS^t/— KibcfigffiOs ecTableTU 

seqTable [produc 
e r I d ] (Dt^'^iy^ s e q 2 6 . 4 0 \zMH^h 

p L a b e 1 ^ x< — /l- Kirproducerld 
y ^—jV Kfi t e s t__p k t U-^;^ 2 6 . 5 0 F^JC 

-acptf^a2 6. 1 3 Ofi. rc^y— K 20 

ixttL5;6^if s e qTa b 1 e*:5j:I>* 

C R c 31 7 rS^:;^^s^^N ^ 5 o 

- s e q MfrfraH 2 6. 1 2 0 Ji. y Y (D^MrO: s 

e qTa b 1 ^r\-^^m^^-WM\^f\^^t^}£0t^h^ ^ 

-ack genfill2 6. 1 1 0 fi. a c k Sltri^r 

^'^i^i'^'t^o 30 

'd) ^ i^is'^^'^y h • X — i57(^r equest in 

(2 6. 10) esponse_in (26. 

2 0) ^^^tt^iJl^tci^^^btT^'Kb^n^o Tt/fL. 

[0 0 5 5] m2 SA^XUm2 8 Bfi. ^it^<^ y h 

[0 0 5 6] 11 2 9 rec_pkt_reg2 
6. 6 0\^<D-^yi^ * y^-jUh^t^(hmmy<^y h (* 

<ttl. request_in ^#■^^T?lJ 2 6. 1 0 t r e 

s p o n s e_ i n^$-ibtT^iJ2 6 . 2 0 t^^^MlZ^-t^ 
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[0 0 5 7] H 3 0 Xit. seqTableru-r2 

6. 30^seq_reg26. 40 {Cl^^iAti^^ r e 
q u e s t_ i nt#^fT3?iJ 2 6. 10 ^fcti r e s p o 
n s e_i n^^-^tT?ij2 6. 2 0 l^<^>^<>>- :y h OS^^^I* 

[0 0 5 8] ms 1 xit. n^n^^i^m^^^^m-^mx^ 

-tfjii^-h. request__in26. 10i^fz\'Xre 
s p o n s e_ i n 2 6. 2 0 ^kzmz'i^'^^ffm\:'t 

mmy ^-^i'\''^m^'r^t^^fci,'tMm\:^^y hi:v 

[0 0 5 9] da 2Tii. rcoy — Ktcr Ki/;^fi^$ 
thtz^ity<^y h(D^m,x\ cRc^m::m\:.x. mm 

y — K-^#^/Sii^/N'^ >^ h ^±f^ request_ 
i n ^^'hffH2 6 . 1 0 * fcf^ response_in 
n^nm 2 6. 2 0 rtO^N^ y V ^^Stt^^Si-^So 

[0 0 6 0] [US 3-C(^, CRC:3i^— ^tS{c:iS;CT. 

113 2xa:^^'^j:t:>^^mz.^m'r^fz^\zi^cD^'^jt^c^ 

[0 0 6 1] ms 4 xit. ^mMJf^m^fz s s o^^^ 

[0 0 6 2] ms 5 T-fi. -'N'^^ y h (Dt^Tm^i s e q T 
a b 1 eTU^ m(DW.^^i'¥i:mz^'^i-^^^'^tzaW 

m^m±-r^tzisb(D^i^(D:^m^y':^ h-t^o sffci. 

m^J} C R CitStC J: o 5o 
[0 0 6 3] m 3 6 T f^. .'N"^ h7&S;ft-gb)i|li^T')l|iJ?# 

n^thtzssoxh^^^\zi^<Dm^w.^^±f^^mff'r 

[0 0 6 4] 113 7Tf^. iH^t>'^-^r:y bCOSJl-TB^tC, 2 
6. 7 Of^tc:fr<acRC;efSfe^;6>if ^;&^^-7";:^ h-T^o 
CRC7!>^^^^^-rfet9. s e qTa b'l emih^^^^^Jt^ 
nTV^/^V^:®'8', s 6 q T a b 1 e 5rM^i-^o 

[0 0 6 5] 113 sxit^ fi&<7)y- KfdT Ku-;^fg^^ 

tl./c#:^J^:;^/^<^ 5/ hm<Ds e qTa b l eMff:^^ — 
[0 0 6 6] H 3 9 Xit. y h ^^^mW-CJiLli^^ 

^tlfzn^}t^^y<^y hm(Ds e qTa b 1 eWmy ^ 
[0 0 6 7] 114 OJLUP^CD0-|J(i. ^IgB^CDi/^XJUCOijtl 

114 0\^y.mt. iyiy:^^^XW^'hnm^(Dy<>!ry hco 
r: t (Di'fm^l / - Kcot^tte. ^ ^ 7^ :y y r t (d^ L . 

tm±Lfzm^(DS SOfpT4v'>7i^'r> h=^'^>hlz 

njt^Lx\^'^^-tyf<^:^^^^o 



17 

[0 0 6 81 ^mcot^mmxm.tim^^^t. * 

i 

b 
r 

b 1 
r 1 

X 

d 

BRC 
CRC 
SRL 
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1 n p roc 

i^m^ : a c k_b u s y. ffiffi q^/l-- y^S^tr) 

C R CWU.njU—y'^<D'^ 1 (Dy^^ y h 
a c k^fz{t^m^(0\^^irM^ 
ack done 

c R c ^ -Sil^^t/^— :r 

s e n d_p k t :/ 



[0 0 6 9] 1 3 9 4. 2ja*&m^%itUfc:*:^P^<7^if 

IEEE P 1 3 9 4. 2\:im^< V >:^l^^y h ' h:^ 
^i^X^±y<4y^^>it^titz.y<^y Vmi^^-^y^- 

-to rco^-g-. y<-(yv-(>ity<^y hi^. ^^Mumm 

(SSO) ir^^T-TxVTKT^V h • =r-7V K(^1^ 20 

1 3 9 4. 2>^-r • b:^Kn2i/(;iJ; ^Jte55-r^r 
[0 0 7 0] - i^^:*rzXi.^fi. ^^fflcp^:!!^^ . 

^'thtL^\:im^fmmf)j^\^'^'^^-V^^^ CRC 

Rcin^- • y<^y hcDiE'^f^wnnmi'^^m^-r^j 

[0 0 7 1 ] $fc. yiyp^X-S SOCD^T^'f^:^ 

hmt^^i^^-t^ X^J:k^\^^ 06 4B) 40 
:y h • yyt—^y h^^i-^teO^o h^iyi-t I EEE 
P 1 3 9 4. 2 ,i:coFp1(7:):7''y :y v^cDlS:f+7ji^:^(|ii(c^*t 
it^nsr t t)^E^nfcv\ fci:x.(i> IEEE13 

94-19957 — y^^t^(D 2 KB -7^' a :y ^ ^^OSiA-^^ 

t^TWi2-^t\^xmm't^:::.tt^x^^ I EEE i 3 
9 4 - 1 9 9 5y<^y hCDW^m^^y'V ^yiy±X^n^tl 

[0 0 7 2] s soiiij^{^ctt7:>=';^ h(i. y>'>5^^'u:yh 50 



±X'^^- h ^ri^y- kst^ci^? 2^w h^^ot^sgi. 
TJ^uv>;^:^{cs-5v^T7^:-ro ]) >'^'uy hm(Dm^mjs. 

i-Xe 3 /-Vfj:(DX\ rttidj: :ft::^cl 2 8 BStrS 
SO t^^^-^v-V) *i;^vif-h-r5 

[0073] IEEE P1394. 2 C7)^^?t^«t^ 
IEEE P 1 3 9 4. 2-t^J:i:>'hiyVTJi^'::c^::x^y 
IEEE1394-199 5'tfj:t>'hi^ V T Jl^ 

^-Cfo5o r(7:>:7'n h ri/bfi^ I EEE 13 

9 4- 1 9 9 5 t?:)*:*: =1 > K • :y h<^r1^?K~- h Lt'.^ 
7!iS^. 1 6 B<Dy<^y h^XXJ^e 4 By<^>y MdJ: 

[0074] IEEE P1394. 2(1. V V 1^ y 

i-^, m-(DSjl.\^^^it^tzit.Jt^^y<^y h (JL^Tt^^ 
^1*5) (I. -^0/<-r>'"?;^ • ^<:y:7r;65-«■^T^Ii^.CV^:& 
^icy- \^izXoX y iy^^\^y fp^ictfAl-^:: <^7!)ST 

= 6 4B) . rcor — ^^^^^^i-^^J^plHgiCl/^^ t <!: tic 
^ (t:) J: u ^ t CO { c Tt^ o T I ^ 5 o 
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[0 0 7 5] y<^y h ' y^ — ^v hfi. y<^y h^gg 

[0076] IEEE P1394. 2 T/a h riyl-fi. 
[0 0 7 7] S S Olliff#(t : 

[0 0 7 8] hS7->^{biS±t^<^ • 

t;iIEEE1394-1995 X^m-^m^^mX ^ 5 30 
;»«;t;:=iXi^OlOf^, fc i: X. 2 K B CO;^^ V ^/<>*r h 

[0 0 7 9] rcoj: p7t^;^:*r-XAfi. D /J^ V> 
(< = 6 4B) Y^mK.fz.Y;^^ i-fj:t>'h. 

[0 0 8 0] P 1 3 9 4. 2{ZX^S SOijtm 

[0 0 8 1] {$¥mm. *f^) ^- Kc7:)#3-7> K(i. 

ft«^^5fey- Ki(Cj:oT^?T^n^o TiUftj 50 
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[0 0 8 2] &LTcDm,mxi-i. ^(oy-v^mity<<r^y 

h^mi^i-^mmmy-}''xh^:^\ ft^y<^y hi:m 

ihi-^^Litmy— 'YX:h^^\ ^my<^y h^fzi-^ft^^ 
y<^y h^tei^i-^y y yiy-^fz^-tys^ • J—YX 

fzm:i r{>^^fi{|ijy- Kj 

[0 0 8 3] i||it^N->5r 

V V ^^—y<jvf^ 16t'';/b<7)rtargetl 

< . 5/v>*fcfi^-Y ^;/^ • Kfi. target 

0 u r c e I d J :7 ^ — K(^^ laKfy— K-(0:^^n— 
[0 0 8 4] ^(7){tilC7>>'^— K(^. n— y 

hJiT* * rO/"^>r^y bSr^-ftL/cn o d e I d ^rgffiS'J 

i"^ (6 t:^^'>' hcO) Tp r o d u c e r I d J :7 — /U 
.Kct. rcO/N>r:y h ^P^^^tCHS^^'Ji"^ fci^tC Tp r o d 
u c e r I dJ J — KlCjzoTfy^:) STibn^ (8 \f y 
Y<r>) Tp L a b e IJ :7^— /u KTi^fe^o p r o d u 
c e r I d y p L a b e 1 y ^—)\^ YVtll^ 

bt. ^—ioj\^'^):y^^\yyVt^\zm'^^f\.fz^.^^^ 
7t<b;^fi. y<^y Yiy^y^^ y^^X^X.^^n'^^fx.^m 
-^{^Sfd^T;65?f:b^5o fp r o d u c e r 

1 d J :7 >f —jv Kir TpLabe Ij —jv Kfi, ^ 

K (C ct 5 St y S T t ^t ^ o 

[0 0 8 5] «Ttc:tBa!i-5-?:Ofdi(^:7^-/vKt LT 

itSlJ) . sourceldy - YicM l^X^COlt^^^m 

m\^mmi- ^fzi^{zft^^y<^y h{zxtixm^ti^i^'u 

'-y<jUfj: r t L a b e 1 J ^ H\ ^(Dy<^y hffl 
<D^ y y t v"— 1^ (D?^:^^^ts 3 2 \::y h(DCRC=^ — 
YXhh c r c 3 2 y >{—JV Yt"-^^^ 
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3 
2 



2 1 

4 6 

I I I I I I I i I I I I I I I I 

targetid | 

tLabel | | 

sourceld | 



( 



10 0 8 6) s:itmy-h'\ci.-ox±^^fi,tznMit^^ 

3 2 1 

2 4 6 

I I I I I I I I I I I I I I I I I 

I localBus I producerld | 

I c r c 3 2 

[0 0 8 7] n^lt^^T h^u::^^i^^ir^tzmz^ yzCO 
-/hOTtargetldjTK • ^7 ^ —fV K 

(i. r 1 o c a 1 B u s J itlgiJ^ 1 ) h 

jt<0/<^y Vt^h(0^ \fy y<Dp roducerldy' 

fflOc r c 3 2 /l- Kf^. CCldfStti-^S SOp< 

[0 0 8 8] n-:;!7yu . y bg^;ii5^-;^f^T* 

[0 0 8 9] ZCOStfi. P 1 3 9 4. 2 y h 

OS:$:!if'^coS^i:/^oTV^5 (fcfcL. Pi 3 9 4. 40 

*7t(^■^c7:>-R^j^::^i^c7:>l^-fi^7)^c7:»/c^56^C p reduce 
r 1 d y ^—)\- p L a b e 1 V ^—J\^ Yo:>W[')5-k^ 

[0090IP1394. 2:/Dh 



0 0 

8 0 

I I i I I I I I I I I M I 

1 I^^T^I I =i-K I 

I pLabel | | producerld | 

I of f setHi (addr) | 
offsetLo(addr) | 

c r c 3 2 I 



0 
0 

I I 
1 I 



0 
8 

I I I I I I I I I I I I I I 

\ ^^':f\ I p L a b e 
I 

[0 0 9 1 ] CRC^^-CO^-^, m^lT-#^tit-CDlt 

J^^e^o (n-;^/!- ^ J - ^ ^ J^T ^ 
J—Y\z.X,<>X\^o:>\^yVt'^'^m.-t 
tCfE«-r ^ S S 0)i|lJ?#(ti^^- h • ;^ tr^XM.<DW}i¥ 



[0 0 9 2] ^im\ r^ffl^^j (omi 



[0 0 9 3] v-:^7^AO|i^ : Kfcioit^gE^flCO^ 

p L a b e \ y ^ —J\^ Kirproducer I d:7^-- 
/I- Kt^y h • v^TS S OHli^^^Tl-tt ;^ =^ 

Xi. ^iSi- ^ (C-i^ ffl T # ^ i: V ^ 5 ±^f^ 
[0 0 9 4] r (7:>*'g'OS*m;t;fi. :y S y > 

:fcJ:t/y Vi^^^u^y MiHffm^.tt$g 

&te^-r5:i t tv-^p 2ocD^^v^;^;!!l^^pItgT'fc^o 

SAn/<^:y ^^iS:^t^PJJ7t'{t;^)^^E^iLTV^;5o 0T'.i^-r 
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+ 



I 

+ 

[0 0 9 5] ft^ftiJ y - K : 

cOiilf h • ^ — >'l- K : # p r o d u c 
e r I & t^h(r>mt^X.Xllt^'^^<^ y hiCH-TSp L a 

be 1 :7>r— /I- K^{tfflLT3o(Oi^:7^':7-<— /ix K (tA 
pLabel. sso ( 1 :y h) 

pLabel.bzseq ( 1 t':y h) ^ S S O^PplCOfca^ 

pLabel. seq (6 \f y h) 
[00961 fpLabel. s % o \ \f y V\%^ Z.(0 

y VmcOT Y\y:^'^f^'h^'^SOm^\'^^ (s so = 
1) X^^'^t'^^f:.\%^o'^^^fS:\'^ (s s o = 0) ;6^^K) 
fi^l-SSi-^o Tb z s e qj "T^c^ * jlHi? t:' :y 

O^SS-r^fcij!>{c{^ffl-r^„ 6t:^':yhO FseqJ y 
>^^K{:i. rcDp r o d u c e r I d;6>b^r?$bJfe^"^<$^ 

[0 0 9 7] y h±cD^p r o d u c e r I d 

^(^p Label y Y'^^'^\^X^(0^ J — 

Ytmw^^mm^^mnx^^i.o fct-^ p r o d u 

c e r I d(Ci:oT?**^o -^<7:>tictgt»$g^ p r o d u 
c e r I d^^^^C?:*^-!! p Label y ^—J\^ K^lrbgSi" 

ss omm-^\'i^mn't^ ^ o \ziy<^ y 

[0 0 9 8] rgij^j y - Yiy^ 6 >y h co 

^\ #^iJ^^^^;SL7t:icoy — K/0^^cost3^v^^^T 

[0 0 9 9] S SO^rplTOii^ibflFTi^. ^Ll/^#i^ 
TpLabel. seq:7-< Yt^=^ C/zl o 6 4 "C 1 



3IM#J < 

I 

? ACKlCM"r-S)X^-? 

I 

— >S«#J + 

J: o T ^ f^^ L b ?i v \ 
10 [0 1 0 0] ^T(^{^ffli-^ rcRC:3i^-j 

ffi^O^fTIE:^!^— ^{ifrt.'g-tftCD^ LTaA?-r^C 

[0101] CRC^^- - }\^—^ ' V±y h : 

[^(D-t^X<D/— YXZ(Dp roducerld t^BiL 

o d u c e r I D{cm-t^^V >^^ l^y h • y — Kc7:> S 
20 S 0^^«gtt$6f^. rcOp r o du c e r I dSfc(^fl!l<D 

f^^Sfiyy- Kdr^-r^ s sot/u^-^^ ^ Vi/^rsio^H 

[0 10 2] mSril^-r^fci^tc, ;^7^XA(D 

1. V l^i^^^i^y h±(D^^iBmy— hl^. ^—:tf/h"]) 
>^'Uy h±(D^^iBy— KfflOo— ;^/bp r o d u c 
e r I d{zX^X-f l^r^yi^ y^^^^nhtlfzSSO^m 

2o£7)liLli*#tt:7^— /i^K^^-^ti^o cne>o:7^ 

2. CRC^^-^1ttb-t'^#p r o d u c e r.I d 

• yN>r:y h J: «9 ftjOftm^T^^^ S S OliliJ9^'f+ 

40 h^m^tirzm'ik(Dy<^y h^xm^. -mcDssom 

h\^X^mM}¥^—^':^0>h^m±-t^tzMz^ c 

IC. CRC=^y-m(Dm^ik(Oy<^y h^Xm^(Oy<^y 

^ ^ - Wi^^T^P-:/7)^^jE^I^T Uz^T-t^^ X\ /u 
50 n bnyi/;:)>^^--^^si-5Ci:;i>>"et^o CK^WIi?;!)^^ 
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[0103] s^tffiy y - K. : ^mm y - Vit. ^<dss 

0'ikmmm^^lE\^\^^m^\::.. ftgOp r o d u c e r I 

m-Jix V±y hm\Cp r o d u c e r I dtCj:oT^ZJ 
SatS:^^n. p r o d u c e r I DiClioTCRC^^ 

[0 1 0 4] §:1t<iy/- Kfi. CRC::Il^~i^{t^^°^:y 

[0 10 5] LtS^L. ^fflffiijy— Kfi. i^hi^^(Dp r o 
ducer ID;^)^bCRC:^^~ - y<^y bic:^<^^ 

5o ^(OXomU^m^<^^y h(D^r(.^'thi^. ACK_ 
e r r o r <Jr LT#^iJ;«i-5PfltcSmiJT'«iai LtES 

seqTable 

seqTable 
seqTable 
seqTable 
seqTable 
s e q T a b 1 e 

[0 10 9] S SOii^^Tii. seqTable:7>r~ 
/t- KSri^T^OJ: 9(-ffiffli-5e Tver i fj 37 — 

K(^> ::(75s e qTa b 1 e 7!)^«0M«|^^$nfcr <t <i: . 

producerl d T^^'^^Si^^^ V — ^ i^cOf^ 

-^j-^^^o r s s o J 37-1- Kfi> s s omm-^^"^^ 

-9-^— M-5fci^(crcDp r o d u c e r I d y — KcO 
(11^6^) ^®tg5r^^-r5o abe^{-i:^(l\ p L a b e 

1 . s s o \fy hji. ft^myy — Ym<Dr vxy^^^mm 
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tLyy^y h OW^EiHT^fc^^-^tC S S OJi|li¥#tt1f ^- 
KT*(i. 3f-^C0 producerld ;i>^bCO|| 1 <D^<^ 
h:^i^S^tT^ ::c7:>p r o d u c e r I Dti^h 

10 [ 0- 1 0 7 ] s s o(Dm^(D^mm(D^^mj'^. ^^m^ 

doneioiO'ac k_b u s y ^<-^r:y h^^MX^^j: 

[0 10 8] :Lti^mf^-t^tzi^. #^iSffiijy- K(^. 
20 JWTO 6 o<?:»:7 ^ — K^^;^-. producerld 
(CctoT^ Vt^':^^;^;^^^^^??^^^^. seqTable 



i^-fCDfzi^COp Label T s e q J :7 ^ — ^t^t^^ 
Tb z s e q J t^'-^/ h(i. ^1 cO^^ffi nf>/N/r :y h 

pLabel TbzseqJ t'tvb J:b®5 ^ ft^o ^ b 

Jt^^^iZp r o d u c e r I d(^5§:^$ix. ^itmy<y 

(^^h<) v±y h^ti^o mmzi. Ta c p tj y 



verif [producerld] (itf^yb) 

sso [producerld] (lt:":yh) 
seq [producerld] (6b":yh) 
bzseq [producerld] (lt:':yb) 
busy [producerld] (lt:":yf) 
acpt [producerld] (6t:'':yh) 

^J:¥i'^Xit. m\t^<^y Y(D Ts e qJ y ^—jW^t<r> 

it'^t^'j^^mx^^, 

[0 110] ^{tffjjy-Kfi. ^^T<r^^ofJ:m^\z.■m^ 

Oproducerl D/i^ b^<?^/.fff LV^ S S 0)IK;?# 
{iy<^y V^^\i^'t^Z.tt^X^^o 
pLabel. seq = seqTable. seq [producerld] + 1 
[0 111] ftm^^WiX^^i^^^ seqy 

seqTable. seq [producerld] = pLabel. seq 

[0 112] ^coft!l^7:>J;l:g!^co^.^T^i. WT^#flB.^i^ 

40 fcl/^o 

[0113] wCOproducerld^^^CRC:^^ — 



ti^a-^ s e q T a b 1 e . 



q [producer 



50 



I y ^ —JV K^ii^iStTt^ pLabel. seq jfifC !J 
-^y V-^K^o Cltlfi. JtUTOi ^{-rcDprodu'c 
e r I dt^ho:>^<^ y hS/cl^^(Dp r o d u c e r I 
d s e q ^ Y'k)kM't^ r i: {C 

if {pLabel. seq <= seqTable. seq [producerld] } then 
seqTable, seq [producerld] = pLabel. seq 



27 

[0 1 1 4] CRC^^y- ' /l^—^(D^m^'^^it. s 
eqTable. acpt[]:7^ —/U K^Cfi ?i < . s e 
q T a b 1 e . s e q iS/f tt/i^'SM ^ tL>5o y hit 

accept: { 



r e j act 



(15) mmW-l 0-164140 

28 

s e q T a b 1 e ^mMn^i^m-T^ CI t ;^i^S^Tfc 
[0 115] Ts e qj ittS^i^^gbr^fc^®-^, y<<5r 

{pLabel. b2seq~=seqTable 
. bzseq [producerld]} 

& {rece iver not busy} 

{pLable. bzseq = seqTable. 
bzseq [producerld]} 
& {seqTable. busy [produc 
e r I d] =0} 

& {receiver not busy) 

&{pLabel. seq>seqTable. 
acpt[producerId]}} 

{receiver busy} 

{pLabe 1. bzseq=s eqTabe 1. 
bzseq [producerld]} 
&{seqTable. busy [produc 
e r I d] 1 } 
{pLabel. bzseq = seqTabel. 
bzseq [producerld]} 
&{pLabel. seq<=seqTabel 
. acpt [producerld]}} 



= '^accept 

[0 116] h^^m^^xit.fj:\^^r^. i^m^wmy/i^ 



T V ^ U ^ t WSJ? Its? ;0^^?J^ L T V 
[0 117] fs e qj it^^^^^^iiX^^m^. v 



a c k 


don: { { p L a 


bel. bzseq'^^seqTabl 




e . b 


zseq [producerld]} 




& {r 


ece iver not busy} 




1 {seq 


Table, busy [produce 




r I d] = 0 } 




& { r 


ece i ve r no t busy} 




1 {p L a 


bel. seq<seqTable. a 




c p t 


[producerld]}} 


a c t 


b z y : { { p L a 


bel. bzseq=seqTabel 




. b z 


seq [producerld] } 




& { s 


eqTable. busy [produ 




c 


e r I d] = 1 } 




& {p 


Labe. seq>=seqTabel 
acpt [producerld]} 




1 (p L a 


bel. seq>=seqTabel. 




a c p 


t [producerld]} 




& {r 


ece i ver busy} 




= a 


c k don 
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iSrll|Ii?Jte?^s^LTV^5^^. y<^y htia c k_d 
o n e i: LTi^m^n^o ^tlh^^^^ a c k_d o n t L 
m'^Jt^^tlfj^^^^m^. y<^y hita c k_h z y t 



[0 119] 



(16) m^W-l 0-164140 
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[0120] rbusyj:7^ —yU K ^ T a c p t J y 

s^o Ts e qj ^m-A'-^^xh^m^. ^m^wu^ 

tt^^x^^, 

pLabel. bzseq ^= seqTable. bzseq[producerId] 
7t/dL. Tb z s e qj ^^-f— yUKfi. =^y—W^^^^ 

{pLabel. seq > seqTable. acpt [producerld] } 
[0121] ^l(D^nrfy<^y h7!)mtti^tltzm^. 
s e qTa b 1 e (D^^B it\iiT<D X 5 tCfS:^ ^ tL6o 



10 



c e r I d:^^hmm^t l.Xy<^y h^i^^-t^m^ 
(seqTable. busy [producer I 

d] =1) ^n'^'t^^tKx^. ^<r> -^m 

V (Dm^^^-X seqTable. acpt[]y^^ — 

%\(Dm.m^^<^yyii^^^^^fh^:^xy^)-^ 

{ seqTabel. bzseq [producerld] 
=pLabel. bzseq 
seqTabel. busy [producerld] 

= rece iver busy 

seqTable. acpt [producerld] 
=pLabel. seq } 
[0 12 2] fz.fz\^^ ^ 1 coS^tr^N'^^^ ^/ hf^. ±tg(7:> ^ra#7!)^b1^tti^;Jx5c 

if { {seqTable 
= 0} 

& { p L a- b e 1 



pLabel. bzseq = seqTable. bzseq [producerld] 
[0 12 3] s e qTa b 1 e 0#3S @ fit^T<7) 

busy [producerld] 



seq>seqTable. acpt [ 
producerld] } } then 
{seqTable. busy [producerld] 
= rece iver busy 

seqTable. acpt [producerld] 
=pLabel. seq 
{ 

else/* '^\(D^m'^^<^V Y-^fz\%::^=7—^Wfl}\--''Zf */ 
{no update 



} 

[0 12 4] mk\z-^ #y-Kf^. ^T(r>X.oi-Si'^^\z 

producerld t^hW^^^y v 40 

pLabel. seq > seqTable. seq [producerld] + 1 
[0 12 5] ^^{t^ rt7)yN°^^y htC(^*S(jCRC75^fc 

^^t^/<^^y htCOl/^T(i. seqTable CO^Jjg 
[0 12 6] > \^ y V S S Om^^m^n^'^ - 



$B ;6 ^ ^JI ^ 15: ^ ^ 5 ^ ^ If . L 1 1 ( i 7i b V > o 
[0127] ^iimA'^mm . s e q T a 

b 1 e '^^'J]W^%'t^tzt>\zm.n^t\^fz s e q T a b 1 
^•^myfy Y^^'a^mih^m\%^<^y h^cJ:oTf■7^^- 
V-t^Z-ht^X^-^^ )^T(D3<:>(Dy.y^yy't^^!j^.mXi:> 

-So 

1 . s e q T a b 1 e []T U-f (i. mM^^^:t>'{zfj:^ 
7!)\ *fcfii~^TC0 producerld {;iOV^T s e 
qTable. varif[]=0 i:^'^ i.X V ± y h ^ 

2. ^^s SOT Yi^y^^mi^cD (*fi^) y^tn^y.i.zX 



31 

v)^ u — ti^i- • y — p r o d u c e r I d^Sr^glS 
U> ^tl^i^^O seqTable. verif [pro 
ducerld]=l <SrS#i2^tro i" v e r i f J t:":y 

3, seqTable. verif [producer 
I d] f)^^i^mit^tlfz:li;km'V\ puroducerld* 

if {seqTabel. ve 



(17) ^m^l 0-164140 
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* it.ZCDy— KCD s e q T a b 1 e iZT Vl^MB'^^thtz 
(1. ■^(Oi^—^^^y h ' T (seqTable) {Z 

[01281 seqTable iA^ tt^fflfi, Jl^ 

rif [producerld] 



= == 1 } then 
{seqTable. sso [producerld] =1 
. seqTable. seq [producerld] 

= [current" seq" from this produ 
c e r 1 d ] 

seqTable. bzseq [producerld] 

= [current" bzseq" from this pro 

ducerld] 
seqTable. busy [producerld] =0 
seqTable. acpt [producerld] 
= [current" seq" from this produ 
c e r 1 d ] 



[0130] JbiecO-J: 5 fj::bm'^mm't^ t . Wi^(0 V 
[0131] ±m(OX^fj::^m^^ mtR^ hm^j- 

[0 13 2] it^m^m 

[0133] IEEE P1394. 2 S S O HH 40 

}) vi> ( I E E E 1 3 9 4 ~ 1 9 9 5 /.e if) 
IEEE PI 3 9 4. 2 y — K--Oy :7 h 'i7:x 

1-5 s so^xyi^-efo5 chti^^n^o so 



[0134] S S OJiii^^+tt;^ ;^^XA{i IEEE P 
1 3 9 4. 2 (^(^^. ^f^) i^-b';^iCJSffl^ix5<^ 

5:bttTii7:^v\ r.co7<;^7^Xi^fi. ^^mtOTKi/;^^ 
PBl;6^ibOfiES03-v:/ K • ;^ h y — A^i^7^^— h s 

> K^^^»ffi-e# 5 i: ^S^-r^c 
[0 13 5] S SOT K 1/;^^Pb^ (S/iif^S SO Kp<-r 

» TD-b^^fey- K-^cDsi-iffirpTT-s s omm-^^-^^mn 
^^'S^^mt^^h^m^. s so K^-^v • :=j-<'>Kfflco^ 

5ty- K;^^S SO|ilIi^#if^tgT?icttnfS?i?:>-r. 

[0136] f-Tjf- h idi^^^^i IEEE P1394. 

2t7:)^SM 

Trc7:>p«;t7^XA:^ I EEE PI 3 9 4. 2 CO:^S(c 

[0 13 7] m'^^^^mx'h^i)^^ mm\z.^^fz^\c\ 

E1394-199 5^(Dy^V :y h-T^Z-c*^ 

[0138] mmz.-r^fz^(D^— vi\L(D'^m\'X. 

Xj y ^—J\- rACK_e r r o r J 



33 



Normal, non 



Type field:0 
-S SO 

1 Extended, non — SSO 

2 [reserved] 

3 [reserved] 

4 ACK_d one 

5 ACK_b u s y 

6 ACK__mo r e ;^ h • 
^tT7i^i£i^S*/*7 ACK_e r r o r */*S SO^— K 

[0 13 9] y i>(Dfci^(D=i — y^'itcOitm-^'it. X 
:y h^/cfil 3 9 4{CX^ 2KBy<^y hit. &.T<D3 

i^T^^^yV: •i6^h<W:^':/^y^X-h^. 6 4B 20 
[0 14 0] y^^J yiy[ZX. 9P1394. 2 V l^^^Uy 

y V (i1S^C7> 6 4 B (D^Wl t 1 0<7) 6 4 B OS^^ ^ 
'M^^fv^) o U;O^L. /ci:;t(i. 1 3 9 4^3fey-K 

y<^y Vi)^h 1 oc7);f^^V^/^>r^y V^m^iLX't^\:i 

y h ^wm.tLXir^ t t^x^ 5o 
[0141] nmT(DmM 

mx^ ^"f-^ ^m^Ki:\^^{^i:(Dy<^ y h (mity<^y 



(18) ^mW-l 0-164140 
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^:zt^mM-r^h(Dtm^i-^^Xi'±ti:<. J:«9— ^ 
JiftS^^^ 7 — MtB ^ ^ X A tf Tfc -So 

h^ ^ i^(7:>9j)m<D x^^mi'^cDtzi^ic^ i^m 

[0 14 2] mm-^^ hV-YVI^cT) r^fflcfij SK^ff^ 

i.tz:^^xit. i-tm^9cy-h^x(D y^m^] 

[0 14 3] CRC:^^- ' y<^y h (miBy}t^^y<^ 
y ^S/-c^iACK<DV^-ri^7^l^) (iftS^^i^i^ffffljy- K 
■C'^tB^n^o s soii|S;^## K^-OT-f^. :i(o^m 

fi. rutli^^i^— J (^^jUT-^ httJt-ACK_e r r 

5/cai^>tCACK_e r r o r ^i^jli- 5 V^ ^ Kfir^r^g 

^tL7tS^O^ft?^/N°<^:y h^-t:\ -f^XCD^m^S S 
ft\i\ so Kp<-r V-eCRCi^^ — ^SMl^T-rSf:: 

[0144] 

[0 14 5] ^ — ^ 1 : y — KA;:)>by — KC-^tO^g 
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(19) 



^mW- 10-164140 
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: B I 

pLabel.seq: 20 

pLabel.bzseQ 1 



C I 
21 



21 
I 



B 

22 



.21 bzy 



C I 
23 

1 

23 



I 



B 
24 



24 



C I 
25 



25 



->- 



|< — bzy;i/— >| 

« 

...23 bzy— ' 



Done 
Acpt 



Done 
Acpt 



seQTable.seq[producerId]: 

19 . 20 21 22 

seqTable. bzseq [producer Id] : 

1111 
seaTab I e. busy [producer Id] : 

0 0 0 0 

seqTable. acpt[producerId] : 

19 20 21 22 



23 
1 
0 

23 



24 
0 
0 

24 



HCOfil : 



Packet completion: 



Accept/reject: 



Bzy 
Rej 



seqTable. seq[producerId] : 

19 20 21 

seqlble. bzseq [producer Id] : 

111 
seqTable. busy [producer Id] : 

0 0 1 

seqTable. acpt [producer Id] : 

19 20 21 

[0146] ^cDm^^m-r < <:>-^^(Dmn. y<^y h 



Bzy 
Re] 



22 



21 



Done 
Acpt 



23 



21 



Done 
Acpt 



24 
0 
0 

24 
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>>-:y h 2 lioJ:I/2 3^|f#-^#ttUfc (pLabe 
1 . s e q) y<^y h24:JoJ:t/25ir i.XWmit^^ 

T(7:^^<>r:yh ( (7:)#-|jT'(l^^<>- :y h 2 3 ) fi. m KD 

r^ffl^»J ^;il^lf^^T^tb^ { fb z s e qj CO 

igM(-J:oT:^^n5) r a c k__b u s yj 

^fzit. ^<Dy<^y hit Ta c k_d o n e J ir LTtg 
YKiy^J- KB{c>?fLTS S OHij¥#^ :y h Sri^fS 



(20) #r?^^l 0-164140 

38 

producerldy — Kid J: ^^SKtr^'i— T?" 

CD&a^feti, pLabel. bzseq y ^—J^^ KS: ( 1 

• /— K(c:ii^^ix6o r(?:);^i7>;/:/(^, '^c;o s e q T 

able, bzseq [producerld] MS:^ 
ItpLabel. b z s e q iritfef ^CirtCj: t9s 

10 — ifeL/^l'^:^'^. s e qT a b 1 efi:(ij^T<7><J: 5 

seqTable. bzseq [producer I 
d] = pLabel. bzseq 

y h$;Jx^o 

«» 

seqTable, busy [producerld] 

= 0 

-r^X(D{^(D (#§:{tfflij) KiO:^^. s e q T a b 

20 le. busy [producerld] (iO \Z.WC^ $ 

[0 14 7] CRC:^!^— SP:^T^7:>^|J 
^ffl ^fT:h)7il/ ^ C R C ^ ^ — 2 0£D 

[0 14 8] <5r— ^ 1 : K C T^^dti $ tL7t 7 - K 



(21) 0-164140 
39 40 

aaife : 

Bl CI B| C| B| C| B| C 
pLabel. seq: 

20 21 22 23 * 20 21 22 23 

pLabel.bzseq 

1 1 1 1*1 1 1 1 

* 

CRCX^-StSfr: 
20 



20 



H >. 



H A<^)S«fi : I < y\,-rf > I 

ffiffl't'lfta: ...2 1 • 

y — HBODfi: * 

i^^v h^T : * 

Done ♦ Done Done Done 

Acpt Acpt * Rej Rej 

* 

seqTable. seqCproducerld]: .* 

19 20 21 22 * 23 20 21 22 

seqTable. bzseaCproducerld] : * 

1 1 1 1*1 1 1 1 

seqTable. busy[producerId]: * 

0 0 0 0*0 0 0 0 
seqTable. acpt [producerld] : 

* 

19 20 21 22 * 23 20 21 22 

* 

y-HCOfil: * 

Packet completion: * 

(CRC) Err * Done Done 
Accept/reject: 

(err) Rej * Acpt Acpt 
% 

seqTable. seq [producer Id] : * 

19 20 20 20 * 20 20 21 22 

seqTable. bzseq [producer Id] : * 

1 1 1 1*1 1 1 1 
seqTable. busyCproducerld] : * 

0 0 0 0*0 0 0 0 

seqTable. acpt [producerld] : * 

19 20 20 20 * 20 20 21 22 

[0 14 9] -10l2^iJ{cr^l-r5l/K 07!)^cDa#:i:iI : CR I^T:lT:bn5o Z.(0\n'X:\%^ 2 1 O C R C ^ ^ -T^^^^tti 



41 

::;rt(c:*l-LT. y - KC^^^^•<:^:y h 2 l ^<ttH 
•v^-f. b/a c k 2 2 jo J: 2 3{c}i|i;^ 

^{c:te^^ixfc^c7)jm/^jiS7:)S2 0Tfo5:6^bTfo'5o C 

U >y h • y — K^O seqTable. seq [pro 
ducerld]:7^ —/U KP^T^HLli^ :^ ^7 V h ^ (20 

V\ ^IHlJjy — KC7:> seqTable. acpt [pr 
oducerld] :7 ^ — yV K ^ tDl:bt^^(c£<5v^T. ^ 

5o h^coHIIffS^i: tie, pro 

ducerldt^pLabel {Z.7^tlX^^(D b z s e 
q M^aiffl i--<TcD seqTable. bzseq 



(22) 10-164140 
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* [producerld] S^SE^Pf^ A^^^fc^S(CiS:^•r 
^o seqTable. bzseq [producer 
I d] M(iy-tr :y h KO s e q T a b 

le. busy [] — KtseqTable. a 
c p t [] :7^-/U'Kfif*}#L?t^{ttt*f/.e^?tev\ rtt 
?5(d:7^-/IxK(^. r.^7:)^5feffl^7:>IELv^i^;^a c k. ^ 

10 :yb2 2Sr^ESL. y — KC >^ h 2 1 t 2 3 <:7)p) 

^=7—t^fi:\^^^. y-~KBiCti|l|lj^#-^ 
[0 15 0] -ir-;?; 2 : y — KAT^^ttl ^ ttfc y — K 

20 



(23) 
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B I C I 
pLabel. seq: 

21 22 

pLabel.bzseq 
0 0 

CRCX^— SKfT 
21 



23 * 

0 * 
t 



■+I 



B 
21 



21 



C 

22 



23 * 
0 * 



^m^l 0-164140 
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B 
24 



24 



B 

25 



Bzy 
Re] 



... 22 err' 

. 21 bzy — 23 bzy- 



Bzy * 



Rej * 

seqTable. seq [prodac er Id] : * 

20 21 22 * 

seqTable. bzseqCproducerld] 

0 0 0 * 

seqTable. basy[producerId] : * 

0 1 1 * 

seqTable. acpt [producerld]: * 

20 21 21 * 

Packet completion: * 

(CRC) * 
Accept/reject: 



Bzy 



Re] 



23 



21 



21 

0 

1 

21 



Done 
Acpt 



I <— bzy;i— > I 



Bzy 
Rej 

22 

0 

1 

21 



Done Done 



Acpt Acpt 



23 

0 

1 

21 



24 
1 
0 
24 



(24) 



45 

seqTable. seq [producer Id]: 
20 21 21 

seqTable. bzseq[producerId] 
0 0 0 

seqTable. busy[producerId] : 



21 



seqTable. acpt [prodQcerld] 

20 21 21 * 21 

[0 15 1] :L(Dma. ^<^y h;i^-^(^^$t-r^{tAtL 

V^(DT\ ^<^v h2 1 i: 2 3li:^*^?3 t^^?ip Lab 
el. seq :7 — /U- K^^L. ACK d o n ej^;#* 



21 



4#B^¥l 0-164140 
46 



* 

22 * 
0 * 

21 * 



23 



24 



23 



24 



21 

[0 15 2] ^-;^3 : 



(25) 



47 

B I C I 
pLabel. seq: 

21 22 

pLabel.bzseq 
0 0 

CRCx^-Httfr 
21 



I 



->- 



23 * 
0 * 



-Hi 



B 
21 



21 



C I B I 



22 



23 * 24 
0 ♦ 1 
* 24 



^mW- \ 0-164140 
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B 

25 



I < — x^- • )v-^ — > I 



...22 err* 

, 21 bzy — 23 bzy- 



I {—hzy)V—zf-) I 



Bzy 
Rej 



3» 

Bzy * Bzy 



Rej * Rej 

seqTable. seq [producer Id] : * 

20 21 22 * 23 21 

seqTab 1 e. bzs eq [producer I d] : 

0 0 0 « 0 0 

seqTable. busy[producerId]: ^i^ 

0 1 1*1 1 

seqTable. acpt [producerld] : * 

20 21 21 * 21 21 

* 



Bzy * Done Done 



Rej * Acpt Acpt 



22 * 

0 * 

1 * 
21 * 



23 

0 

1 

21 



24 
1 
0 
24 



Packet completion: 
(CRC) 
Accept/reject: 



Done 



Acpt 



49 



seqTable. seq[producerId] : * 

20 21 21 * 

seqTable. bzseqCproducer Id] 

0 0 0 * 

seqTable. busy[producerId]: ^ 

seqTable. acptCproducerld]: * 
20 21 21 * 

[0 15 3] Z.(Dm{^. J^Ti7>-ig(7)*^(cS-5< tcO 

[0 15 4] Z.<Dmx\X. CRC^y— l?t^:?TyL.— y^tc 
A5t. pLabel. s e q fiI*Di;tJ«y — KB CO s e 
qTable. busy [] K^seqTab 
le. acpt U Z7 ^ — KiClS":^'^ "^T. • /l^ 

seqTable. busy [] 1^"^ h^^^^^tlX^^^ 
^(DX\ y<^y ^t"it§^^t^5o ^tz. s e q T a b 1 
e . busy [] tvTV^^?!)^. (p L a b e 

1. seqfiStseqTable. acpt D y — 
/V'Ki:Oitg?^c:S-^V^T) t^ibh(Dy<^y hh^m^ 

n^^^j (D-^-ii^m-^nhtixy^^^cDx. wurfi.fz 

y<^yh2 1 t23\tt^-h^h ^^m^j tLX-^^Jt^ 

[0 15 5] i^m^^^—y't'^mif^m^rMh^n^t. j 

^ KA;^^ seqTable. bzseq [prod 
u c e r I d ] :7 —/U YlzX^ ^^:^^^^t>th. y 
h 2 l;05/<<>-ty h 2 4 t UrSeiH^tL^o y-KBIi 

WXhhi)^^ oi)^\:Lm^\^^Xs y<^y h 2 4 *S:^•tA^^ 

hf\.^i)-i:ot-^wm-t^o :iz,xm'&'r^mx\t.^ s 

mji^t^y^^y h 2 4*5 J: 1/2 5 7!»5if •i=5b'bSttA;h.<b 

[0 15 6] m^mz::r.=y-^tm:^ir^m^\^\^m^n 
m^^Siit^ti. mjti.xm^^nh^x. /u-y(Dm 

*&Srt$or ci: T^fc^, ZM'^:J^^^Xh^m^^t. pLa 
bel. h z s e q):fy h(DV^m^^^i-^ t-r^X(D 



(26) 



21 21 22 
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23 24 



21 21 21 * 23 24 

y — K • ^^^y h^^^^(D seqTable. bus 

Y U y ^ KirseqTable. acpt [] y 

^—/i^Y^V^y h't^y)^hX:fo^o Z,tih(Dy^—/u 

- KA;6^a^.L7t:^^(c:PSbtL^o 
[0157] i'ff^.my- K : ^*:^#-^tT^iJt^S 

3. V >i!^m^ich<>X. V :yy\^y h{zMl.xmn^ 
ri^it8im^(Dmit:^XXfm^y<'yyh^^^^j:^ Tsen 

30 d_p k t J n^'iim 

4. S^n-r-<#ffiffl4'^^">^?/ Vt^hfii^ Tr e t r y 

_pk tj ^$*.^T^lj [cnti:ttffl^'^KtT0^5£?i?«§{c 

5. r e t r y_p k t ^^^^T^lJ^?E±^ 
nm(Dy<^y s e n d_p k t ^-^tr^UtC y 
/l-i"5/ci^(7:> Tr e t r y_p k tj l-i^y^^^iSM : 

1. V >i:^m\H{::.h^XZ(Dy—Y^(Dmitm^Jt^^^^ 

2. h^>'if^i/3 VgP^fwfeoT. iJ^^^^oTV^S 

[0 15 8] h^^f^'i^^^C/H >gP^(7:)^{tj|JJ^m*Dj:D^ 

Jt^^y<^y h(om^. m^i^cx^xi-xn-hn^^i^m^cD 

[0 15 9] S SOllli¥#{t^— YX(D:i(Dy— 
50 ^^my<^y hmmit. 'i^-hfj^^iXitfj:<ts^L^m^t 



(27) 



1^10-164140 
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[0160] S S Omm^if^mX-(OJt^^y<^ y h<Di^ 

mm 

IEEE P1394. 2T-fi. — K(Cx — 

[0 16 1] SiA^^jt^^>5ry Kti, SiA;9^x-^7!i5^ 
[0 16 21 ::^7)tSiS7i^e> 2 0(7)MPp^;6S^^-r^, 

[0 16 3] :^^i/3 >'ilrt(7:>*iA^ h^Vif 

[0 16 4] SiA^ i^m(7^^2(Di^mm 

[0 16 5] ^it/^"^:y h ^^-i^j^T^'Jffiit 
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P 1 3 9 4. 2{Z.JtCtlT Ki^;^^Pp1{1. -^(c. Jl«i^ 

#it ^ n 5 m-g- 1> fontf. ^ n^cf V t ^ 

(7)-t:\ y-Kfi. m:^(DT Y^Uy^m^'ev'y^-^mm]^D- 
Send_pkt S S Offiit. ill^t^ fcf^iS^^N^r h^fc»9 1 

. send_pkt. val (l) m^^PJ) */ 

10 send^pkt. sso (1) /* =r-^^' K • T Kl^;^^^;^^ 
b */ 

send_pkt. done . (l) /* ACK_done$:S<tx i^^^:;^^^^^ 

send_pkt. busy (1) /* ACK.busySrS'ft. SKfT# 

lb */ 

send_pkt. init_er (1) /* =i^y —W^^J\^—Zf^(D^J} 

m (mi(D) h */ 

send_pkt. no_xmit (l) /* — • jV—zf^\Z. ]) -^^ 
Uy V---(DiB^m^^\^^/ 

20 send_pkt. err_ret (l) /* ^ ?f^?T/l-~'7'P^O:3i 

send_pkt. seq (6) :L(Dmit/^t^^y<^ y h m 

CO)IlJ*#-^ */ 

send_pkt. packet (N) /* N tf y Y (D^iny^it^<^- 

V */ 

[0 16 6] ffi.'gci. sen d_p k t GfiF I FO# 

l/^<o rack_doneJ ^ LT#*J^:^^nfc^^t 

[0 16 7] s e n d_p k t F I FO^-^btf^yt^lSD 
ft}lOt#"ibtT^U. U'>^:^^$'. MUX. i^tg-^v-v. ^ij" 

[0 16 8] cRc^^- • ^v—^mcDi'^mmj— k - . 

;0^fc^o To r_J r|iLii^##j ^ 

or_last_ct (6) /* xn—ij/V' ^) 1^ y V \Z. 

mit^Mzmms s omm-^^ p k t ^cohlw:^ 
*/ 

no_last_ct (6) /* n— • y^/i^'U:yh(C 

i^m^^/c^Sr^S SOUii^^t p k tm(Dm^io^ly 

Y */ 

50 or_rcvd_ct (6) /* n— y 1/ y h /^)^ 



(28) 
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* ^7 > h(D~^n^— h 



*/ 

[0 16 9] ' y l^^^'Uy h{Zy<^y h^i^il^ 

^tb^t^ h ' >'<;^T' sen d_p k t Ui^^:^^ 

F^l-feit$n6 send_pkt y ^ --jU ]^^t^^-t^ 
X 0 fC, ^(Dp Label y ^—JV Yt^WL%^^^o — 

pLabel. sso = send_pkt_reg. sso 

pLabel.bzseq = bzseq /* ^"n — /<yH^^^tlJy — 
t^Sgt: :y h */ 10 

pLabeL seq = send_pkt_reg. seq 

p L a b e 1 . s e q ^*OfiS7jiSfiJ ST 7!»^(i. 

[0170] ^Ittty^-f Visits? : JfrSL^v^r^ n 6 4 * 

if {pLabel. seq <= seqTable. seq[producerId] } then 
{seqTable. seqLproducerld] = pLabeL seq 
} 

3. KJi. WX'(r>^^\z:^ p r o d u 

pLabel. seq > seqTable. seq [producer Id] + 1 
[0171] ^v^:xP64;^7^7>'^i^J:oT \ X'O^l 



1. S^tWJ/- Kli. i^TO:^-^tC. FjT^t7:)p r o d u 
c e r I &t^h^9))fi:%^\\^^^SSOm^¥i^^<^yV^ 

pLabel. seq = seqTable. seq [producerld] + 1 ;^>^0 
pLabel. seq > seqTable. acpt [producerld] 

2. rtDproducerl d;^)^CRC^^— • /l^— >^ 
^^^TLrv^^r.i:^rcoy— K;?!)^^1taii-5ir. seq 
Table, seq [producerld] — 
K(^?EtTO pLabel. s e q y ir :y f ^ih.'So 

s e q :7^-yi/ K^i:b^-r5::<t(cJ:oT^^n5o 



^;i?'cev^r t^p reducer! d J: oT5^ll^^ Lfc 
JWTtO:^'^fi rA< = Bj 30 
J[^TO:^'&(t rA>Bj 

[0 17 2] ^miifm. 

[0 17 3] ::.(oxof^f^m(r> 0-^(0% \<Df^mr(:\%. 

CRC:x.^— s e q T a b 1 e [produ 40 
c e r I d] y h-f^r 

Oi--<TC0/— K^t^ s e q T a b 1 e. seq [pr 

o d u c e r ID] t^COy ^—^^ Kit. y > U 

[0174] [W)0#(C. seqTable [] cD-fllfi, ^ 
r CO / - K(cP^1-r 5 PS "9 IE LI/ ^ t a^c^ ^ ttHff/^ 6 

x^t^v.. ii^jnijjy- KcbftucT^y ^i^^M^u/ 50 



[0175] L/c;05oT. p r o d u c e r I d y — K 

[0 17 6] cc7)Pn^lii->(+t-^¥*J6^<^«m«fi. 
j^-f^r t {cj: 40i^/i— :7'$^II^Ti-^ p r o 

ducerld/ — K;^^'^^::^^— • ^1/— :7" ^ ^ ^^i" 5 
Zt-rsh^o producer IDy — Kft. ^tt^i^^ 

y ^ Kit. r^7>producerI D^^O s e q T a b 
1 e. s e q [] :7 ^ — /!/ K7!>^^fpl LffilClS:^ ^ tbX V^/^ 
ttttCf/^b^<^V\ seqTable. bzseq [] y 

L;^^ L. seqTable. busy [] y ^ — K 
(i. rcOproducerl d (D^itM / — K(C 

hmm^itZD^y hl.xafj:h mm\c. se 
qTable. acpt [] y^—JU Kt*^S<7)^ 4 

[0 17 7] fzfzL. m.^u(0^il'^Ty<^y h (^'5!)<t 
XV . producer IDy — K(t. ^Ot^'— ^P^g 
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[0 17 8] ^WJ:ACKr^^i^^ri^^'X:Wi^[\(Dy<^y 

Z.ht^'^^^o L^O^Lx seqTable. bus 
y [] *dJ:I>* seqTable. acpt [] fS:}i(0{^ 
(Dy ^-J\^Y\±. SO^ral^^fflLT s e qTa b 

le. acpt [] y Kt^lffi ( p L a b e 1 . s 

eq) ^»#iAtf/ii:\ {^<0^m-C^^X^)^yV\.fii 

[0 17 9] 5t;05S^tcM-r^^2 0Pp^M-rii. ^<D^ 

^producer IDy — K<D8S9^ ^ OPp^MSrtSo T 

u c e r I D y — Kf^> >' ">5r y h V^J^ U y 

5fe tCliii- 6 ffi^ CO V ^-f tb t:' C R C :ni ^ ^ U 

[0 1 8 01 riOPp^Sfc:*j-r5fi¥i*:»fl. (6 t'^>> h 
(O) seqTable. acpt D ^ K^#§: 
{tfflijy— K<r> s e q T a b 1 e Tl^-r tCiij[JDL. ^(Dp 
roduc e r I &t^h(0 ^^<D p L a b e 

1 . s e q 'k'^ts^^^y hiCioT^CD s e q T a b 1 
e. seq [] ^ ir ^/ h ^ tL^ SutC r CO 7 — K^eft^ 

'a(Of:L}h\Z.^'^^MzM\<0^<^ y (s e 

qTable. busy [] y >( — ;V Y\Z.^^X^^t^ 
5) ^ta^-f-^r t T^fc^o fc/ciL. s e q T a b 1 
8. acpt [] >^>r— >'l-K^ii©l^i-^^^*i. I^AT^T) 

1 . C^^y- YXmk\z.^\^7K^hM:L^<^ y s e 
qTable. acpt [] i: ^ LV ^Ti^^^X J: «9 /J^$^/^ 

V^*1$ttiC0/^<!^:y h(i s e q T a b 1 e . acpt [] 

seqTable, busy U itD'^y h ^tl^ 

2. :3^7-;^fz(i{^ffi^>/N"^>;/ b;0^/^V>®'&. seqT 
able, acpt ^i^i,^TCO pLabel. seq 

10 18 1] [2 4 0-6 OCDWClP^-r^^dEg^^i 

124 0-6 of^. h ^m(Dmm.m'Si:i:^n ^^^^ 
mtrmmy- Y{zm[^xm±\^. crc (^fcfi^c^ 
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_b u s yi^:^^^e^LTV^;5o 
[0182] 11140 — 60 (Dmxa. o r _ 1 a s t _ 

c t itmwc'^i^]^tz\mm^t^<^y ycoio^i^ vw.x 

fcf?. o r_r c V d_c t 

_r c V d_c tffii:7y — x^tt^o 
[0 18 3] Z.(omx\%. SiXm<T^^m^y<^y Y. fzt 

10 •PL\t. 5 4 <t 5 9 7i^ r e t r y_p k t ^$-t>^t^lJp^(CA 

(or^last^ct - or_rcvd_ct) = (23-59) modulo 64 

= 28 

^/c. 2 8 ti-^COM (o r_ 1 a s t_c t - or 
_rcvd_ct) OL#V>fii:i:^LV^/^>^-^.tLj: 
V\ 

[l2®(7:)^¥^c^flP.^] 

[121] xy)i^UW^^-y VV-^ ±,<DV i^^^uy hk 
[[2 2] Vk^-ty^^ y^mx-h<:>x^ 

[12 3] 3S;^|il*|ilij?{ttt (sso) (7):^'g^tci|^§<t-r 

[124] Wr^y'^l^i^'rlyVm^-^fzl-X^-^^'YkWo 
s s ocT^^-a-ic^^g^t-rs =3-^-:/ K^;^i-[2'efc5o 
[12 5] (22oy i^ifuy Y^7T^\^. ^^im^^^y 

30 [12 6] (2 2(0 1; ViJ'^'i/y y Y(Dnm\t 

Sr^i-|2T'fc5o 

[[2 7] m2(DV >^l-y hi:7j^l^. f^SiPJjy- KOT 

^ v-g >4'^.-t-[2'C^fo^c, 

ims] ^m^fs- Y\cmi.x:^mm(oi^m^/i^-y'mi'^ 

[[210] ;*:|g0^CO^/^v'A£Df^S1^:7^'v^;^7"^^■t■O 

yyx^^:i t^^-rmx^^o 

[121 1] ^mm(Oi^y^TM.comm'^y''iyy^y'M.tt<D 
[12 12] :^^m(Diyy^y'M.cDt&mi)-yiyy^'rM.t't(D 

W}i^i:7j^'ti^mmxh<>x. m^^^mmxi^) y^nw^ 

yy'xh^:it k^-trnxh^o 
50 [12 1 3] ^^mcOi^y^'rM.(Dmm'^y'iy:^TM^t^<D 
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yyx-h^Z.ti:7v^^mX'h^o 

y'yxh^:Lt ^TT^-rmx^^o 

[mis] :^^m(Ot^:^^M.(D^m^zf':y^'TM.t^(D 

mYf^Tj^-tt^mmx-h^x^ {^^<Lk^mmx\^) y^^m 
y-^^x^^:: t^Tj^-tmx^^o 
mi'f^^i-t^mmxh^x^ m-^A^mmxit) ysf^m 
y^xh^z t^Tj^-tmx^^o 
m'f^Tjk'tt^mmxh-i^x. m^A^mmxit) y^^m 

^yZ^^X^^Zt^T^-tmxk^o 
[mis] :^^m^i^:^TJ>^co^m'^-:fi^::^y'M.t^(r> 

y^x^^zt^Tjk-tmx^^^ 

y-y^x^hz t^TT^-tmx^^o 

WiYf^yr^-ti^mmxh^^x. m-A^m^^xW) y^f^m 

yy'XhhZt^TT^-tmxiy^, 

[[22 2] ^^^(Diyy^^M.CDmm'^y^iy^^J^t^CD 

mi'firTs^-tmmmx^'ox. i^^A^mmx^^) ^^-^m 

yy'Xh^:ztt:7r^i-mx^^o 

[[22 3] :$:^m(Diyy^'rM.(Dmm'^yiyy^'rM.t't<D 

mi'f^^'tmmmxh^x . m^^m^xit) y<^m 

yzfX'h^:Lt ^7j<irmx^^o 
[122 4] if^m(r:>i^y^'rJ^(Dt'^m^y^^^y^r'J>^t^(r) 

mY^^TT^-rmmmx^r^x. m^^rn^x^^) y^mi^ 
yyxh^zt^Tjk-rmx^^o 

[122 5] :$^mmcOiyy^r'J^(Dt^m^y^i^y^'rM,t^(D 

mi'f^^i-t^mmxh^x . {^{L^^mmxit) 
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yy'xhh:! t ^Tjkirmxh^o 
[122 6] :^^m(Diyy^'rM^<Dt^m'^y'iy:^'rM^t^(0 

mi'f^^'ri^mmxh^x . (^^A^mmxit) y^nm 

y rfXh ^ r t ^TF-f-HTfo 6o 

[[22 7] :y^y :^-:^o:>mM.i)^h^^mo:>W)Yf 

^^i-y^u yi^mxh^o 

[122 8] ^ity<^y h • ^-r ^>i;ft:7jki-^^ ^i^^ 

mxh^o 

10 [122 9] g{t/- K{:iP^TLT:*:^9JcOi/;^7^AO|^S 

'\)-'^i^y>'rM.t^(DW}i^'t7jk't?^mmxh^. 
mso] ^my — h^\zmi^x:^?^m(Diyy^TM.<Dt^m 

^y^^^^y^j^t^com'f^^irmmmx^^o 

[123 1] §:my—\'lcmLX:$:^m(Diy:y^TM^(D^m 

'^zfiyy^TM.t^(DW}i'f^^-rt%mmx^^o 

[[23 2] S:m/—hlzmi.X:^^m<Di^::^y'J:.(D^m 

•^yi^y^y^j-^t^<DW}i'f^^-rtkmmxh^o 

[[23 3] — KIcKLT;$:^0J<7D>-x^A(D|^M 

1^ :/ ;^ A ^ CO ib{^ ^ TF-r fi^[2 T fo 5 o 

20 [(23 4] ^my—h'{cm\^X^mm<DiyPsy'M.(Dt^m 

'^y^'yy^y^j^^t'^comi^^Tjk't^mmx^^, 

[[23 5] §:imy — 1.x :^^m(Diy:^y^M. (Ot^m 

'^y^i^:^y^M.t^(Dm'^^7r:-rmmmxh^o 

[[23 6] ^<t/- KtC||LT;*:^0^<^v^>!^7"AO|^^ 

•^y^iyy^y^j^tt(DW]i^^yj^'rt^mmx:h^o 
[123 7] ^my-\'izmi.x:$:?tm<Diy^TM>(omm 

'^zfi^:^TJ^t^(DW}i¥^7j^irt^mmxh^o 
[[23 8] Sr^t/-- K(c||LT:$:^0^cDv';^7"A(^f^^ 

30 [[23 9] S:ity-\'lzm\^X:^mm(Diy:^y'M.(DB^m 

[[24 0] :$imm(Dii-i^ L^^mmmm\:i:i^^-f ^y<^ y h 
(Dm t ^Mthco ^ ^-ty^ ^ y^mx^^, 

[[24 1] :^mm(Oiyys'rM^cDW}mcX^m4 OCDy^u 

y ^ m(Dmm^m^7r^i~mx^^o 

[[24 2] :$:^m<oi^y.y'M.(Dmm::^^m4 ocoyu 

y m(D^ikmvtm^^i-mx:k^o 

[[24 3] :^mm(Oiy7sy^M.cDmmzx^m4 o(Dya 

y ^ m(ommv^m%:7r:'rmxh^o 
40 [124 4] ^mm(Diyy^'rJ^(Dmmcx^m4 0(Dyu 
y ^ mco'ikmikm^Tjk-tmxh^o 

[[24 5] :^mm(Diyy.y'M.(DmmzX^m4 OCDya 

y ^ m(o'ikm^mi:7r^'rmx^^. 
[[24 6] :^^m(D^yy^'^d^(DW}mzx^m4 ocoyu 
y ^ m(D\km:m:^.^7v^'tmx^yb. 

[[24 7] *||0^cDv';^7^i.(Dtbj^(;iJ:'5[2 4 oco^a 

:y ^ (2 ^ ^ *t S ^ ^ (2 T 5 o 
[[24 8] *ll?^cOv';^7^Acoi[]<^tCj:§[2 4 0t7>yn 

y ^ m<D\^m:m^^^^'^mx^^^ 

50 [[24 9] -^^mo:>'yy^=rJ^(Dm'^\ZX.hm4 0(ny'u 
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[ins 1] ^^md^^^::^^ M.<omm-^ 

[(25 2] if-^'m<D^y:^^ hsCDm^V^ 

[115 4] :$:|§9JC0v':^"7"A(DSfj^^(; 
[1115 5] *^P>^cOv-;^7^A(7)lj!jm> 
[125 6 1 •^^^F^(0^y:^'rJ^<Dmm-^ 
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* yi7m(r>%mim^7f^'tmx^^^ 

[U5 7 ] ■^^^n(D'y:^'T J^(Dm\^\^^^mA 0 0:7^0 

y ^ m<r^wmM^7v^^mx'h^. 

[1215 8] 1^W^^(D^y^'TJ^(Dm'^\^^^m^ 0(0^X2 
([115 9] :^W^n(O^y:^J'J^(Dm\m^^^mA0(r>^^ 
[116 0] *^P^<7)v';^v"AcOljbf^iCj:^[a4 Ot^yn 
:J:^I114 O(ozf^ 10 [[D6 1 ] ■^^^^(D'iyy^'r J^(r:>m\^\^^^^A OCD^u 

1 — 4 yV^/Ui/hl 
A — D J—Y 



ct5ia4 0<7>yD 
±^114 0«7:>r^n 
±^04 OCT^yn 
^-51114 0(^:7*0 



.<t5IEl4 0(^yo 
-J:S[24 OO^^^n* 



[HI] 



[da] 



- sso 




A (read or write) 
B (read or write) 
C (read or write) 



X fread or write) 
y (read or wrilc) 
Z (rc*d or write) 



A (read or write) 
A (read or write) 
B (read or write) 
fi ^csd or write) 
B (read or write) 



Z (read or write) 
Z (read or wrice) 



SSO& 4^7<t>;f"-f^h 



A' (read or write) 
B (read or write) 
C (read or write) 



X (read or write) 
Y (read or write) 
Z (read or write) 



A (read or write) 
B (read w writs) 
C(read or write) 



X (read or write) 
Y (read or write) 
Z (reader write) 



(32) 
[112] 
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X 
Y 

Z (done) 



1 





fit 








y-f C<;J¥fflJ ) 









X 



X 
Y 

Z (dooB> 
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[US] 



J — prcxJuMxID I 

} (6-bU Id, thu rin^t) | 

I --pt..ibel (8-bitscnd j 

I 




/-K" B 




fit 






4.it 



/-h' D 



it it 

















1 7yy'> 







Ack 



C31C 



Ack t9 

I -produoerlD 



0-164140 



[me] 





1. 'Jv? • 7-t.>:'giiD<»fe&l.j SSO /rf'T-^h tmuHi^^. 



2. A<^-.|.*s:^ff:l7-?'i»3ti^iJ'7*N; 



E 



(35) 
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1 












/Ac*, i^^jfn 

















(36) 



0-164 



[US] 




SSOput 

^^^^ 






— ► 


'4rCZ<n SSO 




Acki3 OkK^ 


^ 
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[1191 



(busy.selS 
CRC.erf$ ^ 

CRC^intLerr 
CRC^crr ; 
CRC_ock_chk 



CRC_ crf_cnd 



busy_tooo^cmpU 

retry foMts 

H retry-pKt ^ueuc. 

busy_loop_vo) 
busy_loop_pnd 



ocV, ok 



i 



inlLcrr 
no_xmU 



CRCerr 

init_ err 

err_rd 
err^rel 



CRC,ock,chk 
IniLcrr 
ncuxmil 

crr_rcl 
no_xmil 



(38) 



nmW- 10-164140 



(Ell 0] 



9.10 



crc_init_erT = 1?^ — » ^ 



N 



9.20 . . , — 

V-( cre_cir=l? ) — H B 

N ■ ' 

9.30 1^ 

^ (cn;_ack_chk = 1?'^ ^ 



N 



9.40 

v: 




C 



busy_ioop_set$= 1?^ 



9.32 



9-34 



(head of scocLpfiy 
qucuc.vaJ=sl7 



V V 



biLsy_lcx)p_val= 1?^ J/ 



N 



head of scndjla qucue^SC>=l? 



9,160 



9.70 ▼ 

head of scnd_pkt qucucN 



9.36 



SET: 

CRC_acfc_chk= I 
rctxy_pkt_rcg_vaJ = 1 
ack_logic$=Tctry_pkl_reg 
busy_I oop__ vaJ = 1 

hiisy_loop_pnd = 1 
rctry_pki_reg = ackjogicj 
busy_starr=l 



9.80 



N 



bead of ack_qucuc .val = 17 ^ 



9.100 



N 



idle_timc_oul for head 
send _j}kl queue 



SET: 
CRC^init^cn^ 1 
rctry_pkt_n^val = 1 
ack_logic$=Tctry_plct_rcg 

busy__loop_pad == 1 

log^OT =1 




9.90^ 

— ^ ^^^^ ^ acJc^queue .SSO = 



9.110 
J 



head of send _pk:t queue, seq 
head of ack_qucuc.seq7 



9.130 



9.120 
J 



9.140 



^buyy_loop_pDd= I?) 
Y 



(head of scnd_pkt queuc.scq\ 
= c3r_U3t.ct? J 



9.150 



SET: 
biiry_loop_j3iid = 1 
bu5y_ccnplt_set = 1 
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1 ] 



10.10 



SET: 

fetry_pkt_reg_vaf = 1 

ock^logiclvoi = 1 
ock_logtc$.sso = 1 
ock_Iogic$.busy= heod of 

Qck_queue.dusy 
ock_lo9ic$.done = head of 

ock_Queue.done 
ock^IogicJ.iniLerr - heod of 

send_pkl queue. iniCerr 
ock_logic$.no_xmil = heod o/ 

send_pkt queue. no_xmit 
ock_logic$.erf_ret = heod of 

send_pkl queue. err_rel 
ock_)ogtc$.seq = head of 

send_pkl queue.seq 
»^etry_pkt_reg = ack.logic$ 



il<^heod of send^pkt 

\c|ueue.nQ_xmit = 



10.20 

y 



Dequeue heod of 
send^pkt queue 



10.30 



10.40 



SET: 

dequeue heod of send_pkt queue 
dequeue head of ocJequeue 



ir ^10.50 



< ^ffee2_revd,ct = l ^ 



or_revd_cl = heod of 
send^pki queue.seq 



10.60 



10.70 



10.80 



^~<C^itr_c\.> thresho[d_efr 



10.90 



report link failure system error [ 
~^ 10.100 



j err^cl = err^ct 4> T 



10.1 10 



rseLerrlct =^ 1 



SET: 

Brr_ct = 0 



10.120 
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[Ell 2] 



Ack tfe^i 



t r 



1 1.10 



heod of ock^queue.vol = 1 
11.20 



i 



N / idle„Ume_oul for \j 
heod of send^pkt queue = \ 



j--^-^--c^^eod of ock^queue.sso - 



n.40 



M heod of send_pkt queue. seq = 
"n^ ^^Qd Qf Qck_queue = .5eq 



^11.50 

non_order_reg_vof = 1 
ock_[ogic$.val = 1 
ock_Iogic$.sso = 0 
ock_!ogic$.busy. = 0 
ock_lo9ic$.done = 0 
(ock_(ogic$.iniL€rr= 0) 
(Qck_Iogic$.no_xmit = 0) 
ock_!ogic$.err_rel =^1 
ock_logic$.seq = heod of 
send_pkl queue.seq 
dequeue heod of send_pkl queue 
Iog_efr = 1 

non_ofdef_reg = ock_logic$ 



non_ofder_reg_vol = I . 
cck__logic$-vol = 1 
ock_Iogic$.sso = 0 
ock^logtcj.busy = 
head of ock_queue.busy 
ack_logic$.done = 
heod of ock_queue.done 
(ock„logic$.injl_err = 0) 
(ock_Iogic$,no_xmil = 0) 
Qck_logicf.err_rel = 0 
ock logicS.seq = 
heod of send_pkl queue.seq 
dequeue heod of send_pkl queue 
dequeue heod of ack_ queue 
noa_ order ~f eg = ock_logic$ 



0-164140 



[mis] 



1^ 



12.10 



heod of scnd^pkl 
queue^vol = 1 



12.20 



rnon_ order 



heod of snd. 
queue. ssQ 



id«pJ<l V 



3L^ 



12.40 



}^ / heod of ack_queue 
I — y.vol = 1 



> 



12.70 



N / idle_time_oul for 

heod of send_pkl queue 



12.90 



SET: 

relry_pl<l_reg_vol = 1 
oc!<_!ogic$.vol = 1 
ock_logic$.sso = 1 
ock.iogici.busy = 0 
ock_Iogic$.done = 0 
ock_logrc$.inU_crr « 0 
ock_ logic $.no_x mil = 1 
ock_logicj.err_ret ^ 0 
ock_logic$.seq = heod of 

snd_pkl queue.seq 
Dequeue head of send„pkl queue 
leove head of ock_queue 
log_ err = 1 

re lry_ pki_reg = ock_fogiC$ 



/I 



12.110 



head of send. 
queue. init^err 



12J20 



SET: 

relfy_pkt_reg_vol = 1 
ock_Jogic$.vaf = 1 
ock_logic$.sso = I 
ock_logic$.busy = 0 - 
ock_logic$.done = 0 
ock_logic$.inil_err = 0 
ock_logic$-no_xmil = 1 
ock_logicJ.crr_rel = 0 
ock_logic$.seq = heod of 

send_pkl queue.seq 
Dequeue heod of send^pkl queue 
Dequeue heod of ock_que.ue 
log_err = 1 

relry„ pkt^reg = ockJogicS 



12.30 



heod of send_pkl \ Y_ 
queue. no_xmii - 1^ 



i 



12.50 



N / heod o( ock_queue \ Y 



12.50 



heod of send.pkl queue.seq = 
head of ock_queue.seq 



12.80 



jj/heod of send_pkl 
\^ queuciniLerr - 1 



12.100 



SET: 

reUy_pkl_reg_vol - 1 
ock_logicS.vol = 1 
ock_logic$.sso = 1 
ock_logic$.busy = 0 
ack_iogic$.done = 0 
ock_logic$.lniLerr = 1 
ock_logicJ.no_xmil = 0 
ock_logic$.err„rel = 0 
ock_logic$.seq = 
head ot send^pkl queue.seq 
Dequeue heod of send^pkl queue 
leove heod of ocU^queue 
log^err = 1 

retry^pkLreg = ock_iogic$ 



1 



12.130 



SET: 

feiry_pkt_reg_vol = 1 
ack_logic$.vol = 1 
ack_logici,SSO = 1 
ock_logic$.busy = 0 
ock_togic$,done = 0 
ock_Iogic$jniLerr = 1 
ock_Iogic$.no_>:mil = 1 
ack_lDgic$.err_rel = 0 
ack_logic$.seq = 
heod of send_pkt queue.seq 
Dequeue heod of send_pkl queue 
Dequeue head of ock„queue 
log_err = 1 

relry_pkLreg = ock_logic$ 
CRC_iniLerr = 0 
CRC^err = 1 



^m^l 0-164140 



[Hi 4] 



Ack 



< 



1/- 13.10 

heod of send_pkl \ Y 
queue.vol = 1 



13.20 



<gg!h2^3> ^ queue.SSO = 1 / ~| 



U.30 



N / head 
queue 



of send^pkl Vr 



13.40 



SET: 

relry_pkLreg_vol = 1 
ock_logic$.vol = 1 
ock_logic$.sso = 1 
ock_logic$.busy « 0 
ock_Iogic$.done = 0 
ock_loglc$.iniLerr = 0 
ock_logic$,no_xmil = 0 
ock_logic$.err_rel « 1 
ock_logici.5eq = 

. heod of send^pkl queue.seq 
Dequeue heod of send^pkl queue 
relry_pkt_reg = ock„iogic$ 



r 



13.50 



SET: 

relry_pkl_reg_vol = \ 
ockllogic$.vol = 1 
ock_logic$.sso = 1 
ock_logic$.busy = 0 
ock_logic$.done = 0 
ock^logicl.iniLerr = 1 
ock_ logic t.no„5(mil = 1 
ock_logic$.err_rel = 0 
ock_Jogic$.seq = 

heod of send^pkl queue.seq 
Dequeue heod of send_pkl queue 
CRC^err = 0 
CRC_ock_chk = I 
relry_pkl_reg = ock_logic$ 
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[Ell 5] 



\ queue, vol =» 1 / ~[ 



U.IO 



£ 



heod of send^pkt 
queue.SSO = 1 



< 



14.20 



1 4.30 



heod of send_ 
queue.lnll^err 



i 



eng. 22 ) 



J * V 1 4.40 

N/ CRC_e(T_relry Vy 

T /-1 4.50 



SET: 

CRC^err^retr/ = 0 
CRC^ock^chk = 0 
GRC_ InrLerr = 1 
retry_pkLreg_va! = 1 
ockllogicS.vol 5= 1 
ock_logic$.sso = 1 
ack_Jogic$,busy = 0 
ock_rogic$.done = 0 
ock^logici.iniLerr = 1 
ock_loqic$.no_xmU - 0 
ack«Jogic$.err_rel - 0 
ock_loQici.seq = bead of 

send_pkl queue. seq 
Dequeue heod of send^pkt 

queue 

retry^pkLreg - ock.logic$ 



•H.60 



SET: 

retry. pkl_reg_ vol = 1 
Qckl logic}, vol = 1 
ock^logict.sso = 1 
ock«logic$.busy = 

heod of send^pkl queue. busy 
ock^logicS.done 0 
.ock_logic5.init_err = 1 
ock^!ogicj.no_xmil = 1 
ock_logic$.err_ret =. 0 
ock^logicf.seq = heod of 

send_pkl queue. seq 



dequeue head of Qck«queue 
CRC_ock«chk = 0 



CRC„erf_end = 1 
busy_foop_snd = 0 
retfy_pkl_reg = Dck_logic$ 
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[IS 1 6] 



( C t'J. ) 



< 



1l 



15-10 



heod of send_pkl 
queue.vol = 1 



15.20 



heod of send_pkl 
queucsso = 1 



t A 15 .30 
^JL/ heod of sendJpklX 
i \queue_infLerr - 1 / 

-^J . 

t X 15.40 



heod of Qck-queue \^ ^. 



1 



15.50 



idleL^Umc out for 
heod of send^pkl 
_ queue 1 



> 



, £ 



15.60 



SET: 

'^etry^pkLreg.vol = 1 
ockl'ogic$.vol = 1 
ock_Iogic$.sso « I 
ock^foglci.busy = heod of 

ock^queue.busy 
ock^logjcj.done = heod of 

ockL queue, done 
Qck^logici.iniLerr « 0 
ock_logic$.no_xfnil « 1 
ock^loglcierr^rel = fosLpklJ 
ock_logic$.seq = heod of 

send_pkt queue. seq 
Dequeue heod of send^pkl 

queue 

retry^pkLreg = ock^logicl 
iog^err = 1 



< 



heod of ock^queue 



y-)5.70 



.sso 



>□ 



15.80 



head of send^pkl 
queue.seq = heod of 
ock_queue.seq 



SET: 

felry^pkLreg.vol = 1 
ockl'ogjc$,vol =r 1 
ock_logic$.sso = 1 
Qck.loglclbusy = heod of 

ock_ queue, busy 
ock_logic$.done = heod of 

ock^queue.done 
OCk_ logic S.init_ err = Q 
bck_logicf.no_5(mit - 1 
ock_logic$.err_fet := losUpklS 
ack_ logics, seq = heod o7 

scnd_pkt queucseq 
Dequeue heod of send.pkl 

queue 

Dequeue head of ack_queue 
felfy^pkLreg = ock^logicj 



15.90 



^15B.1 0 
heod of send^pkl 



queue.seq = erf_losLcl 



£ 



SSL 

lost_pkt$ = 1 



>□ 



-15B.30 



»osLpM$ = 0 



-150.20 
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[lai 7] 



1 



16.10 



^/ retrv_pkLreq_vot 

\ =1 



CRC^iniLerr 



16,20 
Y 



16.30 
r 



N ^ 



CRC^ock_chk VY 



1 



16.40 



busy_slofl 



15.100 



16.110 



SET: 

4:l_mux$ = resend$.Yol 
send_pkl_reg_voi = 1. 
resendt.vol = 1 
resend$.sso - 1 

resend$.busy = relry^pkLreg.busy 
resend$.done = retry_pkl_reg.done 
r€send$.inil_err = 1 
resend$.no_xmil = 1 
resend$.err_fel = 0 
resendi.seq > relry^pkLreg^seq 



16.50 



N / reUy-pkL reg 
\ .initlerr = 1 ^ 



16.60 



H / reUy_pktlreg\ 
\ -nolxmii > 1/ 



15.70 



SET: 

4:1„mux8 = resendS.vof 
send_pkl_reg_vol = 1 
resend$.vol = 1 
resend$.550 = 1 
resendt.busy = 0 
fesend$.done = 0 
resend$.iniLerr = 1 
resendt.no_xm»l = 0 
resend$,err_rel = 0 
resendi.seq = relry^pkCreg.seq 
send^pkUreg = resendl 



16.80 



SET: 

4:l_mux$ = resend$.val 
send_pkt_reg_vo] = 1 
resend$.vai = 1 
resend$.sso = 1 
resednS.busy = 0 
resend$.done = 0 
fesend$.inil„err = 0 
f€send$.no.xmil = 1 
resend$.erf_rel ^ 0 
resendj.seq = relry^pkLreq.seq 
send_pk{„reg = resendS 



16.90 



SET: . 

4:1_nnuxS = resend$.vol 
send_pkt__req_vol = 1 
resendS.vol = 1 
resend$.sso = 1 

resendi.busy = reiry_pkLreg.busy 
resendi.done = relry_pkLreg.done 
resend$.inil_err = relry_pkl_reg.inil_err 
r€sendi.no_xmil = retry_pkLreg.no_xmil 
resend$.erf_rel = retry_pkLfeg.err_rel 
resendi.seq = relry^pkLfeg.seq 
send_pkl_reg = . resendS 
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[l^ 1 8] 



Ji ( CRC_erflend^= 1 > -V| 



^17.20 



17.30 



^ ^fetry^pkt.reg,vQl - ^^^^ 



.17.40 



.,^JL^7etry_,pkLreg.err,re< = 



17.50 



retry_pkLreg.no_xmit = 1 ^ 



1 




17.60 


SET: 






busy^loop.vol = 0 
CRC_err_end = 0 





busy_loop_VQl 

CRC_err_end^^busy_loop_cmplt ^ 

CRC^ock^chk 

empty 



(47) 
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1 9 ] 



18.10 



relry_pkt_feg_VQl = 1 



18.20t 



busy^toop_snd = 1 



18.30 



p<(relry_pkl_reg.iniLerr = 1 



18.50 



< ^ felry^pkLreq.bJsy = 1^ 



18.70 t 



SET: 

enqueue resendS conteat rnio 

retry_pkl queue 
resend$.val = 1 
resend$.sso - 1 
resendt.busy = 0 
resendS.done = 0 
resendijnit_err = 0 
rescnd$.no_xmit = 0 
resend$.err_ret = 0 
resend$.seq = relfy_pk!_reg,seq 
free2c_rcvd_cl = t 



i 



18-60 



S£L 

Gcnerole ock^done 
lo tronsoclion loyer 



(48) 
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im2 0] 



I N ^ 



1 



19.10 



non^ordef^feq.voi = 1 



j-^!U^flK)n-order_reg.done = f >^ 



19.40 



<noa_order^feg.err„fet = Q 



19.60 



4:1;_mux$ = resend$.vol 
send_pkLreg_vol = 1 
resendj.vql =1 
resendj.sso = 0 
resend$-busy = 0 
resend$.done = 0 

iresendf.xmiLerr = 0) 
resend$.no_xmil =0) 
rcsend$.err_rel = 0 
resend$.scq = 

no_losl_cl + 1, modulo 64 
no_|qst_cl = no_losl_ct + 1, modulo 64 
send^pkLreg = resend$ 



i 19.50 



SEL 




Generole ock_ 


done [o 


Ironsoclton 


foyer 
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lm2 1] 



20,tO 20.15 

j t ^^ vl,, 

L^K^busy-loop^sett = 1 ^ — * i— < ^ busy^ loop, vol 




20.20 



<oOQsL.ct = or_rcvcLct 



SET: 

retry_pkl_reg_val = t 
busy_cycle„vol = 1 
bzscq = — I b2seq 



20.30 



. 20.40 



L- ^busy_cmpll,sel = ^^^"^ 

20.50 



SET: 

busy_loop_cmpft « t 



/♦ scl condiUon for non-error cose •/ 
/♦ busy_loop_cmpIl .lost polenlio! 
pockel moy be enquened inlo 
relry^pkt queue, »/ 



20.160 



SET: 

4: Lmox$ = relryS.vol 
sen<j_pkL-reg_voI = 1 
retryS-Voi = 1 
relyS.sso =1 
relryS.busy = 0 
relryS.dorie = 0 
fetry$.init_ err = 0 
felry4.no_xrnU = 0 
retryS.err^rel = 0 

relryj.seq = oaJosLcl + .1. modulo 64 
or_lost_ct = or_lasL.cl + I, modulo 54 
Dequeue heod of retry^pkl 

Quene 
free2e_rcvd.cl = 0 
send_pkl_reg = relryl 



20.60 



r TioTO 

N ^ ' — ^ V Y 

I — < v busy^Ioop^pnd = 1 ^ ' 

1 J 20.80 

'\ — 



^/ non_order_reg 
^vol = 1 



-20.90 



<non_order_reg 
.done g t 



I 



j:: 



20.100 



busy«ioop_cnipll = 1^ 



20.110 



^ / retry^pkLqueue 
\ empty 



1^ ^2 0.120 



^ / re lry_pkL queue \ 
~ \ emply / 



♦ ^2a t30 



SET: 

biisy_loop_vQl » 0 
busy_loop_cmpU » 0 
bzseq_inverl ' = 0 



bzseq^invert = 3 ^ 

^ ^20 .150 



SET: 

b2seq_inverl = 1 
bzseq - — i bzseq 



(50) 
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Mux 



[m 2 2 ] 




21,10 



(oulasLct - 
or_rcvd,ct) .GT. 
lhresho!d_ct 



i 



21.20 



send_pkl queue \ ^ 
> threshold^sp 



i 



21.30 



retry_pkt queue 
> thfeshold_rp 




1 



21.40 



response queue 



>threshold_reg 




1 



21.50 



N ^ request queue ^X"^ 



> thresholds reg 



1 



2r.60 



response queue \Y 
not empty 



1 



21.70 



request queue 
not empty 



1 



21.90 



SET: 

busy_loop_$et$ = 1 



1 



21.100 



response! - I 



j_2U10 



SET: 
requests = 1 



(51) 
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im2 3] 



^ requests = 1 



22.10 



I ^22>10 

p — <(^on_ofder,fe g_vol " ^ 

99 10 



heod of requestN ^ X_ 
queue. sso = \ y 



SET: 

4:l_mox$ = requeslj.val 
request$.vol = 1 
requests. sso = 0 
request!. busy = 0 
reques!$-done = 0 
requeslJ-intCerr - 0 
requests. no_xmil = 0 
requestS*err_rel = 0 
requesl$,seq = 

no_losl_ct + 1. modulo 64 
no_lost_cl = no„lQSl-ct + 1, 
modulo 64 
Dequeue hepd of request queue 
send_pkl_feg_vol = 1 
send_pW^feg = requests 



22.10 



SET: 

4:1_moxS = requesl$.vol 
requestS.va) = 1 
requeslS.sso = 0 
request$-busy = 0 
requeslS.done = 0 
requestSim'L err = 0 
requ€st$.no_xm!l = 0 
requesl$.err_rel = 0 
requcsl$.seq = 

no„losl_cl + 1, modulo 54 
no_losL.cl « no_losl-cl .+ I. 

modulo 64 
Dequeue heod of request queue 
send_pkl_reg_vol = 1 
send_pkl_feg ~ requeslS 



(52) 
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[1]2 4] 



Mux 



22.10 



^ response! - 




t /- 

busy_loop_vol 



22.20 




22.30 



non_order_reg_vol 



3 



22.40 



N /heod of requesl\_Y_ 




queue. SSQ == 1 x 

22.50 



SET: 

4:t_mox$ = requeslS.vol 
responses, vol = 1 
response$,sso = .0 
responsei.busy *= 0 
responscl.donc = 0 
rcsponseS.iniLerr = 0 
response$.no_xmil = 0 
response$,err_rel = 0 
response$-seq = 

no_Iosl__cl + 1. modulo 64 
no_Iost_cl = no_losl~ct + K 
modulo 64 
Dequeue he6d of response queue 
send_pl(Lreg_vol. = 1 
send_pkLreg = rcsponse$ 



22.60 



SET: 

4:1_rnox$ = requeslS.vol 
responsej.vol = 1 
response!. sso - 0 
responsej.bosy = 0 
responseS-done = 0 
responsel.iniLcrr = 0 
response$.no_xnnil « 0 
response$.efr_rel - 0 
responseS-seq = 

no_losl_cl + t, modulo 64 
no^losLct - no_lasl-cl + 1. 

modulo 64 
Dequeue heed of response queue 
send_pkLreg_voI = 1 
send_pkl_feg = response! 



(53) 
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CRC ^m^^ 



[112 5] 



24.10 



end_pkt_re9_vol = 1 



24.20 



send^pkt^reg.SSO = 1 ^ 



24.30 



SET: 

Tronsmrt pocket with 
pLobel.sso = 0 
pLobel.bzscq = 0 
pLobel-seq =. 

send_pkt_reg.seq 
producer Id = focol ringlet 
ID for this mode (6) 
Enqueue 5end„pkt_reg 
into send_pkt queue 



24.40 



send^pkUreg.no^xmil a 1 ^ 



24.50 



SET: 

Tror>smit pocket with 
pLobel.sso = I 
pLobe).b2seq = bzseq 
pLobel.seq = 

send_pkt_reg_seq 
producer Id = locol ringlet 
ID for this mode (6) 
Er^queue send_pkl_reg 
into send_pkt queue 



24.60 



Enqueue send_pkl_reg 
into send^pkl queue 



0-164140 



2 6] 



A .LT. B 



y- 25.10 

^A(5) = B{5)> -^ 



y y- 25.20 . 



t /- 



25.40 



p <A(4:0) > 8(4:0) 



25.30 



SET: 

(A .LT. 8) = \ 



25.50 



SET: 
(A ir. B) 0 



<!!L<^(4:0) < B(4:0) 



A 



A .GT. B 



|^A(5) = B<5)> 



25.60 



^ A(4:0)<B(4:0) 



^ 25 .70 
c:B(4:0) ^ 

j 

V y 25.90 



t r 



25.80 



SET: 

:(A .GT. B) = 1 



A(4:0)>B(4:0) 




r 



25.100 



SET: 

(A .GT. B) - 0 



B 



^mW-l 0-164140 
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•26.10 



rcquesl_in 



response_in 



seq VvT-y 



lesLpkl UvT^y 



rec.pkl \^y?-y 



26.90 




26.170 



(56) 
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[112 8] 



producerld 



rec.pkl_reg , hdrO ^ hdrl , hd.2 , hdr3 ^ doloO , , _ , ^ 6oio^^ ^{CRC) ^ 

seq Toble (reod) 



I Uck J 
/hdrOJ 



seq^reg ^ rd ^ . i — H — i 



t 



lesl^pkl_reg hdrO hdrl ^ hdr2 , hdr3 ^ dotoO ^ ^doto15^ (CRC) ^ 



lesl_pkl I — — ■ * 

lesLword.cl \ \ \ 2 \ I i 4 i 5 » i 20 i 
CRC-crr ( * * ) 



Producer 

, AckO . (CRC) . Id . Ackl ^ 
rec_pkl_feg }- ^-h 1 — — h 

seq Table (''eod) (wrile)(reod) 
rd . wr . 



seq^reg h 
compore 



tesLpkLreg i — 



ock^pkl » -> 

CRC_ERR 



B 
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SET: 

rec_pkLreg = 
CRC_check$ 




/Pocket slarl detected VT 
\ in rec_pkt_reg / 



rec_pkLreg.lypB \ 
decode = **commond*' / 



SET: 

lesl_vford_cl = \ 
lesL-pkt = \ 
lesLpkt_reg = 

rec_pkt_reg 
pkl_sizet = decode of 

rec_pkl_reg.code 
end„of_pkl = pkl_size$ 



< 



• rcc^pkl^reg-lype ^\ ^ 
decode = '' ocknowledge* 



N / rec_pkLreg 

torgelld = node Id 



> 



SET: 

OCk_pkt = 1 



]J 

<rec_pkt_reg \ ^ 
Jorgeild = node \6y 

i 

<fec_pkt_reg_code \ Y 
decode = requesfy n 

requesLjn queue \Y 
space ^ pkt_size$ 



SET: 
req^pkl = 1 
my_pkl = 1 



<rec_pkt_reg.code \ Y 
decode = respons^ I 

T 



response^in queue 
spoce ^ pW_sf'^* 



jeue \ Y 
iize$ / j 



y 


r 


SET: 






resp_pkl = 1 




my_pkt = 1 





SET: 

queue full = 1 
my^pkl = I 
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[[US 0] 



c 



lesL worded = 2 



> 



^ seq^feg.sso = 1 ^ — ' 



lesl_pkl_j-eg \ ^ 

.pLobel.sso = 1 



N tesl_pkl_re9 
.pLobel.seq = 
(seq_reg.seq + t) 




i: 



lesl_pkLreg 
.pLobel.bzseq = 
scq^feg.bzseq 




• lesL.pkl_reg 
.pLobieLseq .GT. 
seq^reg.QCpt 



seq_reg.busy = 



lesLword_cl = 1 Y 




SET: 

producerld$ = 
rec_pUl_reg.producerld 
seq_reg = 

seq Toble (producerId$) 



SET: 

inh„pkl_wr = 1 



(59) 
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{113 1 ] 



< ^ tesLpkl = 1 y 



SET: 

ocpl$ = lesLpk^-feg 



^ reg_pkl = 1 



SET: 

enqueue ocpl$ inlo 
requesl_in queue 



^ /'esp_pkl = 1 



SET: 

enqueue ocplS inlo 
response_in queue 



A 



< lesLpkt = ) y 



N / lesL worded = \X 
\ end_of_pki y 



i 



SET: 

lesl_ worded = 

lesl_word_cl + 1 
lesl^pkLreg = 

rec_pl<l_reg 



SET: 

lesLpkl = 0 
lesLword^cl = 0 
queue^fud = 0 
req_pkl = 0 
resp^pkt = 0 
nny_pkt = 0 
inh_seq_wr = 0 
inh_pkt^wr = 0 
ock_donc_gen = 0 
ock^busy^gen = 0 
ock_err_gen = 0 



B 



(60) 



^mW- 10-164140 



ims 2] 



< ^ lesLpkt 1 



<lesLwofd_cl =V 
end,oLpkt / 



£ 



CRC_ err = ] 



SET: 

Work Ihrs pocket in 
response_in os volid 



^ resp_pM = 1 ^ 



SET: 

Mork this pocket in 
response. in os volid 



SET: 

Generole ock_gen$ for 

CRC generate including fiefds: 
ock_gen$.lype - 

encoding of 

(ock_done_gen,ack_busy_gen 
ock_err_gen) 
ock_gen$.producer}d = 

producerlD^gen 
ock_gen$,pLobel ^ 

pLobeLgen 



mmW-l 0-164140 



[113 31 

< ( lesl^pkt = 1 ^ 1 



< ^ ie$Lword,cl = 



£ 



i Y 

<rseq_reg,sso = 1 V— — , 

♦ 

^ / lesLpklJreq A ^ 




lesLpkLreg 
.pLobel.seq = 
(seq_reg.seq + 1) 
/♦Add modulo 64 




.pLobel.seq .GT. 
seq-reg,seq 



-^queue^full - 1 ^y>~ 




.pLobel.$eq .LT. 
seq_reg.ocpl 



i 



"^1 3 . 

— -/^seq_reg.busy = 1 \ 



SET: 

ock_done_gen = 1 
producerId_gen = 

lesLpkLreg_produccrId 
pLobeLgen = 

tesLpJ<!«reg-pLobel 



SET: 

QC k_busy_gen = 1 
pfoducefId_gen ^ 

lesL pkl_ r eg .prod uc erid 
pLobeLgen = 

lesLpkCreg, pLobei 



I 

SET: 

ock_err_gen = 1 
produce rld_ gen = 

lesl_pkl_reg.producerId 
pLobeLgen = 

lcsLI<l_reg.pLobeI 
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([[ocklgen logiT^^ 



[ma 4] 



lesl_pkl_reg 
.pLobcl.seq .LT. 
seq_req.ocpt 



± 



lesl_pkl_reg 
.pLobeKbzseq = 
seq_reg,b2seq 




<seq_reg.busy = 1 ^ 



iesLpkLfeg 
.pLobel.seq - 
seq_feg.acpt 




SET: 

ock_done_gen = 1 
producer(d_gen = 

l€sl_ pk(_reg.producerId 
pLobeLgen = 

test_pkl_reg.pLobef 



SET: 

ock_busy_gen = 1 
producerId„gen = 

lesl_pkt_ reg.producerld 
pLobeLgen = 
tesLpkLfcg.pLobet 



(63) 



m^w- 1 0 



i3 5] 



< ^ tesLpkt = 1 y - 



< 



lesl_word. 



Let - 2^ 



— — ^seq^reg.venf = ] ^ 




lesLpkLreg 
.pLobel. seq= 
(seq^reg.seq + 1) 
/♦Add modulo 64*/ 




lesLpkl^reg.pLobel.seq 
.01. seq_rcg.seq 



> 



SET: 

fnh_seq_w = 1 



N / ^ — \ 

"•^ — C seq_reg,sso ~ 1 > 



H/ tesLpkLreg \ Y 

.pLobel.sso =1 



SET: 

seq_updole.verif = 



1 



seq^updote.sso = 1 
5eq_updole.seq = 

lesl_pkl_reg.pLobel.seq 
seq_updole.bzseq - 

le5l_pkLreg.pLabel.b2seq 
seq_updole.busy = 

seq_reg.busy 
seq_updole.ocpt = 

seq_reg.ocpt 
prodiicerld^updote = . 
lesl_pkl_reg.producerid 



0-164140 



[US 6] 



(updolc_logic^ 



my.pkt = I > — ^ 



lesLpW_reg 
.pLobel.bzseq 
seq_reg.bzseq 



[^seq_reg.busy « 



SET: 

seq_updole.verif - 1 
seq^updole.sso - 1 
scq.updote.scq = 

lesLpkLreg.pLobel.seq 
seq_updole.bzseq = 

lesLpW^reg.pLobel.bzseq 
seq^updole.busy = 0 
seq^updpte.ocpl = 

lesl_pkt_reg.pLobef.seq 
producerld^updole = 

les L pkt^ req .producer id 



tesl_pkl_reg.pLobeJ 
.seq .GT. 
seq_reg,ocpl 



lesLpkl_reg.pLobel 
.b2seq = 
seq^reg.bzseq 



UN 



yseq^reg.busy = 1 y - 



SH: 

seq_updole,verif = 1 
seq_updol8.sso = 1 
seq^updole.seq = 

lesLpkLreg.pLobelseq 
seq_updote,bzseq = 

tesl_pkl_reg.pLobeI.bzseq 
seq^updole.busy = 

seq_reg.busy 
seq_updote.ocpt = 

seq^reg.ocpl 
producerld_updol€ = 

IcsLpkL reg„ producerld 



1 



SET: 

seq.update 
seq_updote 
seq_updale 

lest_pkL 
seq_updole 

lesLpkL 
seq.updole. 
— . queue_! 
seq^updole 

tesl_pkt_ 
producerld. 

lesLpkl 



verif = 1 
sso = 1 
seq = 

reg.plobel.seq 
bzseq = 
reg.pLobel.bzseq 
busy = 
full 

ocpl - 
reg.pLabel.seq 
updole = . 
reg.producerld 
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Seq g|/r 

lesl_word_ct = \ Y 
end_oLpkl 




CRC_err = 1 



SET: 

seq_reg = seq_updole 
seq_wr_va! = 1 
rec_ pkl_reg.producerId 
producerld^updole 



A 



lesLpkl - i 



> 



lesLword_cl = \ Y 
(end_oLpkl -.1) 



SET: 

complete CRC check 
sel CRC„crr$ 



< ^ CRCerrl = 1 



SET: 

CRC__erf = 1 



<( ^eq_wr_vof - 1 



SET: 

producerldS = 

rec_pM_reg.pr6ducerId 
seq Toble [^producerld] = 

seq_reg 



B 



10-164140 



ims 8] 

Seq $^fj 



ock,pkl = 1 ^ 



^ seq^reg,verif " 1 ^^/^ 



< 



seq^reg.sso 



<le5Lpkl_reg \v 
.pLobeLsso = 1 




lesl^pkLreg 
.pLobel.seq - 
(scq^reg.seq +1) 
/•Add modulo 64 »/ 



lesl_pkt_reg-plobe!.seq 
.GT. seq_reg.seq 




SET: 

seq^reg.verif = 1 
seq^reg.sso = 1 
seq_reg.seq = 

lest_pkl_f eg. pLobel.seq 
seq^reg.bzseq = 

lesLpkLrcg.pLobef.bzseq 
seq^reg.busy = seq_rcg.busy 
seq^reg.ocpl = seq_reg.ocp[ 
rec_pkLreg.prodLtcerId = 

les l_pkl_ f eg. producerld 
seq_wr:_val = 1 



f^mW 10-164140 



[1113 9] 



(updole5>- 



lesUpkt_rcg 

.pLobel.bzseq 
seq^reg.bzseq 



seq^reg.busy = 1^ 



lest_pkl_reg.pLobel 

.seq .GT. 
s. seq_req.ocpt 




1 



SEL: 

seq„reg.verif = I 
seq_reg.sso = 1 
seq_reg.seq = 

lesL-pW_reg.plobeJ.seq 
seq^reg.bzseq = 

lesLpkUreg.pLobel.bzseq 
5eq_reg.busy = 0 
seq^reg.ocpl = 

lesl_pkl_reg.pLobeKseq 
rec^pkLreg.producerfd = 

tesl_pkl_reg.pjroducerId 
seq_wr_vol = 1 



SET: 

seq_reg.vcriJ = I 
seq_reg.sso = 1 
seq_reg.seq = 

lesl_pkt_reg.pLobel.seq 
seq_reg.bzserq = 

tesLpkLreg.pLobel.bzseq 
seq_reg.busy 

seq_reg.busy 
seq^reg.ocpt = 

seq_reg.Qcpt 
rec_pkt_reg.producGrld = 

test_pkl_reg.produc€rId 
seq_wr_val = 1 
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<^r^W- 10-164140 



1^4 0] 



\ 



rctry_pkt 



4:1 



MUX 



7 



scnd^pkt 



o 



CRC ^f5V 
8/10 ^'K-fo 



i 



o 

or_iflst_ct 
no last ct 



o 



I 



o, lo 



mm 



j 










Ack ^-^^^'J 


1 N-- 











non^ordcr 



Q 



Ack_logic 



or_rcvd_ct 

o 



Ack 



t 



retry_pkt 



o 



rec_pkt 

I 



87l0tS~=7^ 
CRC 4^ 
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[114 1 ] 



rctry_pkt 



4:1 



MUX 



y 



scnd^pkt 



O 



CRC ^J5C 
8/10 :3.^pfc> 



orJft5t_cl 
no last ct 



59 



54 



O, i@. 



o 



am 





1 




23 


22 





Ack /^5^f^J 


^ 











rctry_j>kt 




(21b) 



or_rcvd_ct 



Ack . 



8/10 f'^^h^" 

CRC ;}^^. 



I 



STATE 

•oMELSt_ct: last ordered packet sent 

♦or_rcvd_ct: ordered packet received count, frozen if busy retry required (delected in rcsend logic) 
•Previous busy packets 54 and 59 (modulo 64) arc in rctry_packct queue. 
Note: (or_last_ct - or_rccd_ct) = (23-59) modulo 64 

= 28, and 28 ^ threshold for (or_last_ct - or_rccd_ct) difference. 



•Detect ack_busy in rescnd logic 
•Add packet lo retry _pkt queue 
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0-164140 



[114 2] 



\ 



rfctry_pkt 



or_l85t_cl 
no last ct 



21 



59 



MUX 



send_pkt ^ — s. 



8/10 



o 



54 

T 



scnd_pkt #5#/y 



I 






23 


22 




1 











o. 



non_order 



O 



rctry_pkt 



(21b) 



J 



Ack_logic 



or_rcvd_ct 



rcc^pkt U>S^^ 



8/10 f:h-K 

CRC >^!S: 



STATE 

•Detect rctry_pkt queue exceeds threshold. Set busy_loop.sct$ and CRC_ack_chk. 
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II114 3] 



\ 



\ 1 



4:1 



MUX 



7 



send^pkt 



o 



retry _j7kt 



® 

or_last_ct 
no last ct 



21 



59 



54 



O 



o 



23 



22 



(Delect 



non^ordcr 



(22n) 



Ack^logic .' 



CRC 

8/1 D a-Kfc 



23b 



or_rcvd ci 



retry j>kl 



3^ 



1 1 I Y'^7'7<T' I I 





CRC 






J 









51AIE 

♦Delect CRC error at head of send_pkt queue. Set CRC_err_rctry control bif. 
*CRC ack cfik active. 
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[124 4} 



I 



or_Iast_ct 
no last ct 



21 



59 



54 



1 irr 



o 



:3 c? >'K 



MUX 



send_pkt ^tf^'l 



scnd_pkt 



CRC - 
8/10 n-FYo 



j ^ 








23 


Act jNf?^t>J 


j ^ 








23b 



non_order 



rctry_pkt 



J 



(^n) 



Ack_logic 



or_rcvd_ct 



II I v-^F^-^rnTl 
i . / — s 



8/10 flb-r 
CRC 



5TAXE ' ' 

♦'CRC^ack^chk active. 

^Packet 21 inhibited from retransmission on tlic ringlet. 
*CRC_crr_rctry control bit active. 



(73) 



mm^l 0-1641 



14 5] 




MUX 



scnd_pkl 



cRc f^ffy: 

8/10 <j^K^c/ 








1 v'Jl'y^T 





— ^ 








21 in 


(Dclccl ^ 
Ack ijf 5(f? J CRC err)! X 













or rcvd ct 



Ack 




rec_pkl 



8/10 
CRC 



J 



*CRC-ack-chk reset condition detected through "init_crr" sUtc at head of sci>d_pkt queue 
*CRC_inil_crr to be set because CROenr.rctxy is active 
*CRC_crr_rctry to be reset 
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4:1 



MUX 



scnd^pkt 




(33bh> 



CRC 

8/10 iJ-Rt. 



send_pkt ^■^^i^] 











22n 


I ^ 




1 











or rcvd_ct 



Ack 



rcc_j>kt t^U; 



8/10 ri-K 

CRC ^ 



STATE 

♦CRC_init_ejT active 
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[1114 7] 



\ 



rctry_pkt 



4:1 



MUX 



send _pkt 



CRC ^;5c^ 
8/10 ri^H'fc/ 



or_lasi_ct 
no last CI 



scnd_pkl ^^-^f f^'l 













1 ^ 


23bn 


Ack Ji^^Wj 















Ack . 



rec_pkl |,^>5? 





8/io T^-r 

CRC J^^ISl 






i 






1 fy^'Jr^iT 



f 



STATE 

*Busy retry loop — pending 
'♦'CRC_mit_crr active 

— Tlicrc is no need to maintain "busy** acknowledgment in Uiis state, since acknowledgment on the 

error retry packet will be used to dctcnmin **done" or "busy" packet disposition. 

—"Known good packet'* 21 is transmitted to tJic ringlet and enqueued in send_pkt queue to test 

acknowledgment. 
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[m4 8] 



f 



I 



rctry^pkt 



CD 

no las( ct 



21 



59 



54 



MUX 



Uy^Ly ^ ' 



CRC 

8/10 3 -Ro 



o 



O 



scnd_pkt ;(^*-'^^" 











21 


1 ^ 












21b 


1 ^ 





non_ordcr 



retry _pkt 



J 



Ack_logic 



or_rcvd_ct 



Ack 



rcc_pkt ^yx>' 





8/10 T-^-r 






J 


i 




1 7*Vv77rfy" 1 



'♦'Busy retry loop -- pending 

*CRC_imt_crT is reset by detecting valid ack for "known good" packet 21. 
*CRC_err is set. ... 
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M4 9] 



\ 



retry_4jkt 



4:1 



MUX 



\^fx7 ^ — ^ 



CRC ^/y 

a/10 



T 



i 



or_last_ct 
no_Jast_ct 



21 



59 



54 



Q 



scnd_pkt #^?T^'J 



^ 








22n 


Ack ^t*.m'i 


1 — *^ 











non_ordcr 



Ack_logic 



or rcvd_ct 



Ack 



rc.try_j3kt 



(2lin) 



rcc_pkt 





8/10 -fj^-r 

CRC Jt^% 






i 









STATE 

♦Busy retry loop - 
*CRC_crT Bctivc 



- pending 



10-164140 



11^5 0] 



I 



relry^kt 



MUX 



7 



scnd_pkt 



CRC ^«c- 
8/10 :3-F'-/c^ 



II I >^'n^<f [ 



or_Iasl_ct 
no_last ct 



21 



59 



54 



O 



\ ^ 








23n 




^ 











non_ordcr 



Ack_Iogic 



or_rcvd_ct 



Ack , 



retry_pki 



rcc^pkt l.5/;>j? 





8/10 7^:3-:-^' 




J 








1 



STATE 

♦Busy retry loop — pending 
*CRC_crT active 

-Packet 21 is cycled through but is not resent (no xxnit state) 
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^mW- 10-164140 



I 



0i 
1 



4:1 



MUX 



scnd_pkt 



CRC 




8/10 







rcti7_pkt 



orJast_ct 
no last cl 











21in 


1 ^ 


Ack i|f^'<f5'J 














or_rcvd_ct 



Ack 



rec_pkt ^v";^^ 



I 



8/10 iF^H^ 
CRC jt± 



3 TATjS 

*Busy retry loop — pending 

*CRC„crr is reset by detecting (in Ackjogic) "known good" packet 21 with ("init_crr" and 
**no_xmit") state. 
*CRC_ack_chk is started 
* Packet 22 is retransmitted 
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[|g|5 2] 



retry _pkt 



i 



oMa5t_ct 
no last ct 



V V t 



. O 



All 



MUX 



scnd^kt #^Jrf^'J 



scnd_pkt 



CRC >f 
8/tO j-K^Cx 



1 ^ 








22« 












22d 




11 I rv79^r\ \ II 




Ack. 



Y 



rcc_pkt U5>->.^ 



8/10 f i-r 

CRC .^S. 



T 



ST AT R 

*Busy retry loop — pending 
*CRC_flclc_chk is active 

"Retried packet 22 gets *'ack_donc*'. Remaining "acks" in retry loop must be checked before 22 can 
be retired, "ack done" state is saved. 
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nmW- 10-1641 



[m 5 3] 




I 



CRC 




8/10 













23e 


1 ^ 












23h 



or_rcvd_ct 



II I yyjT^f 



Ack- 



rcc^pkt i^^^^ 





8/10 -fh-H: 
CRC:ft±. 






• i 






1 f^><>'?jrT' j 



?>TAT E 

* Busy retry loop— pending . ' 

*CRC_ack„chk is active 

-Retried packet 23 gets "ack^busy" again. 

-Packet 21 is not retransmitted. 

-Packcl 23, the head of the send _pkt queue, is recogniicd as the last packet by comparing its seq field 
with the *(or_last_ct) register. Its slate is scl to (crr_rct and no.xmit). lis busy acknowledgment is 
maintained with the "busy" state. 



(82) #Pf^^l 0- 1 6 4 1 4 0 



[HIS 4 1 



i 



orJast_ct 
no_Jas( ct 



21 



59 



54 



O 



1 



4:1 



MUX 



7 



CRC 

8/10 J -KTo 



^ 








21in 




— ^ 











non_ofdcr 



rctry_pkt 



or__rcvd_ct 




Ack. . 



rcc_pkt 



8/10 -f'b-F' 
CRC 



STATE 

*Busy retry loop - pending 

*Rescl CRC_ack_chk and busy_loop jnd by delecting '^kjiown good*' packc't 21 with (inil err and 
no^xnut; stale and no error detected during CRC_ack_chk loop. 
-Retried packet 23 gets "ack^usy" again. 

-Set CRC^crr_cnd, residing in rcscnd logic, to detect last packet when it cycles tiirouKh 
rctry_pkt_rcg. . ^ * 
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\ 



1 



rctry_pkt 



oMast^ct 
no last c£ 



22 



59 



54 



o 



4:1 



MUX 



7 



22nd 



non_ordcr 



rctry_pkt 



Ack_Iogic 



CRC ^fSC 
8/10 D-K^o 



or_rcvd_ct 



I I r-yy^-fy I II 



Ack 



8/10 T^^/v*' 



STATE 

*Busy retry loop starts 

— Retry_scnd logic modifies scq field for head of rctry_j)ki to be (or_lasLct + 1) 

—increment (or_]ast_ct) by 1.. 

—Dequeue packet 54 from retry _pkt queue. 

*CRC_cit_cnd active in rcscnd logic 

—Drop "known good" packet 

*Packct 24 assigned new "bz seq" field as first retried busy product 
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[1^5 6] 



I 



rctryjpkt 



i 



or_Iast_ct 
no last ct 



MUX 



scnd^pkl /^"^^^ 
W/^X^ ^ 



CRC i^^' • 
8/10 p-hYc. 











23enb 


















I 5^y7Hy 



24 



T 



rcc_pkl |^5/;z^ 



CRC 4^:6:. 



t'V-^77<7"' 



T 



STATE 

*Busy retry loop active. 

*CRC_crr_cnd active in rescnd logic 

"Packcl 22 completed 

-Packet 24 transmitted to ringlet 

*Busy retry packcl 59 assigned sequence number 25 
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[m 5 7 1 



1 



rea7_pkt 



V V 



4:1 



MUX 



"7 



send_plct 



CRC /^}^^ 
8/10 ^-Rc^ 



25 



orjasi^ct 
no last ct 



21 



<26) 



scnd_pkt ^f^^ffj 











24 


^ 


Ack M^lff 'J 










24d 



non_ordcr 



Ack_logic 



or rcvd_ct 



Ack 











1 



t^*^ 1 



retry _pkt 




rcc_pkt 





8/10 f:b-K 






i 




1 -fyvT^fr I 


1 



STATE 

♦Busy retry loop active 

-Packet 24 "done'* acknowledge detected 

*CRC_err_end loop. 

-Packet 23 detected as last packet with "last packet" sUtc (err^rst and no^xmit) and (busy or done). 
-"Last packet" state resets CRC_enr_end loop 

--"Last packet" state sets "retry jkt^cmplt" control bit for busy retry loop. When 'Vetiyjkt^cmplt" 

is set and the retry _pkt queue is empty, busy retry loop is reset. 

^Retried packet 25'sent to ringlet. 

*Busy rct37 packet 21 assigned sequence number 26 



^mW-l 0-164140 



im s 8] 




STATE 

♦Busy retry loop active with "rctry_pkt_cniplt" 

-Packet 23 in head of rctry_j?kt queue assigned new scq field 25 from (or_Itot_ct) + 1 
- (or_l as t_cl) incremented by 1 
—Packet 23 dequeued 
*Packet 24 retried as ack done 
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"retiy_pkt_cinplt" 
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System for Maintaining Strongly Sequentially 
Ordered Packet Flow in a Riny Network Svstftrrt 
With Busy and Failed Nodes 

Backyround of the Inventinn 

The present invention is directed to a system for support of packet transmission in proces* 
sor-based networks using serial interconnects. In particular, the system of the invention provides 
dynamic ordering support for packets in response to errors in such networks, in particular in ring- 
let topologies, accommodating both busy conditions at nodes on the network and nodes that fail to 
respond or respond for ]ox^ periods of time with busy acknowledgments, due either to overload or 
node failure. 

Serial interconnects in computer systems are subject to a number of different types of ser- 
vice interruptions. For example, when a node on a network croounters a CRC (cyclical redun- 
dantcy check) error in a packet, that packet cannot be accepted. The node that sent the packet 
learns, generally indirectly (such as by a timeout or through the use of idle packets) of the error, 
and eventually must resend the packet. 

Rescnding the packet may not be a simple matter, especially if the network implements an 
ordering scheme, such as relaxed memory ordering (RMO), strong sequential ordering (SSO), or 
orderings of other or intermediate stringency. In a packet-switched network with such an ordering 
scheme, packets preceding and following a packet giving rise to a CRC error may need to be 
resent by the producer node to the target node. 

A particular problem arises when such a packet contains a nonidempotent command^ i.e. a 
command which, once it is executed at the target node, changes the state of that ncxie. such that 
recxecution of the command at that node would yield different results from the firet execution; in 
this case, if the conunand is resent to the node and executed again, undcsired or unforeseen results 
are likely to take place. 

Thus» a system is needed wherein errors in packets can be accommodated by rescnding the 
packets to the target node, while maintaining support for idempotent commands. Li particular, 
such a system is needed that also supports various levels of ordering schemes. 

A particular need is pffesent for such a system that can accommodate ill-behaved nodes. 
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i.e. nodes that fail or take unaoceptably long to reply. In a topology where a single receiver is 
poorly behaved, this node can impede forward progress of the new packets by forcing the repeated 
sending of the same busy packet to complete before new packets are introdxiccd. Thus, a sysctm 
is needed that deals with this potential pitfall. 

Summflry of ihc Invcntiog 

A system is presented whiqh provides for resending of packets that have resulted in CRC 
errors at the same time as dealing witfi busy acks, by maintaining a state at each receive node of 
all known good packets. When packets need to be resent, the local nodes know whether ihey have 
processed the resent packets before, and know which the last known good packet was, and in this 
way arc able to avoid reprocessing already executed commands, including nonidempotent com- 
mands. A busy loop can be effectively suspended while the error loop is executed, and once a 
reordering of the packets takes place, die busy loop can be completed. In addition, the system 
provides for an ill-behaved, e.g. nonresponding node, which docs not send any response back to 
the producer node. Such an ill-behaved node can be effectively neutralized by a queued retry pol- 
icy, wherein busy packets are accumulated in a retry packet queue, until some predetermined 
threshold is reached, after which the continuaUy busy node is effectively removed from the system 
by the producer node ending its atten^ts to retry the packets to that node. In this way. the system 
can proceed with other outstanding packets. 

This application thus relates to a complex system achieving the resolution of three simul- 
taneously occurring problems on a ringlet network: error conditions, busy conditions at one or 
more nodes, and the failure or overloading of a node. A suitable error retry mechanism 
is described both herein and in applicant's copending patent application filed July 1, 1996. entitled 
System for Dynamic Ordering Support in a Ringlet Swial Interconnect by van Loo et al. A sys- 
tem based upon such an enror retry mechanism and at the same time able to handle busy retry 
conations is described in applicant s copending patent application filed July 1. 1996. entitled Sys- 
tem for Preserving Sequential Ordering and Supporting Idcmpotcnt Commands in a Ring Net- 
work with Busy Nodes by van Loo et al. Each of these patent applications is incorporated 
herein by reference. 



-2- 



(92) 



10-164140 



Brief Descriptian of the Drawings 

Figure 1 is a block diagram showing ringlets on a larger network. 

Figure 2 is a block diagram showing a single ringlet, illustrating commands sent and 
received on the ringlet. 

Figures 3-4- illustrate commands sent and received in the cases of strong sequential order- 
ing (SSO) and SSO with nooidempotent requests or commands, respectively. 

Figure 5 shows the ringlet of Figure 2, showing the data structure of send packets and 
acknowledgments . 

Figure 6 shows the ringlet of Figure 2, illustrating the resending of packets. 
Figure 7 shows the ringlet of Hgure 2» illustrating the actions of a consumer node. 
Figure 8 is block diagram illustrating the operation of tiic invention from the point of view 
of a send interface. 

Figure 6A is a timing diagram illustra.ting a busy loop operation of the invention, with 
respect to a scad port. 

Figures 9-25 are logic diagrams illustrating logic subsystems of the system of the inven- 
tion, and their operation, in which (in the informal drawings) hexagons are decision boxes and 
rectangles are steps at which variable values are set. 

Figure 26 is a block diagram illustrating the operation of the invention from the point of 
view of a receive interface. 

Figure 27A is a timing diagram illustrating receive packet timing. 

Figure 27B is a timing diagram illustrating acknowledge timing. 

Figures 2S-38 are logic diagrams illustrating logic subsystems of the system of the inven* 
tion, and their operation, with respect to receive nodes. 

Figure 39 is a block diagram indicating both the structure and the Sow of packets in a pre- 
ferred embodiment of the invention. 

Figures 40-60 illustrate successive states of the block diagram of Figure 39 according to 
the operation of the system of the invention. 

Description of ihe Preferred Embodiment 
Figure 1 shows a standard ringlet I with four nodes A-D, each with transmit and receive 
capability. Requests and responses are sent out by a producer node, e.g. node A. while acknowl- 
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edges (e.g. ack.done and ack^busy) are sent by a consumer node, e.g. node C. Node C may also 
communicate with other ringlets 2, 3 and 4, in a conventional fashion. The present invention is 
directed in particular to solutions for packets that are subject to errors in a ring network such as 
ringlet 1, which can cause unpredictable packet delivery and acceptance, since, although a pro- 
ducer knows what it has sent, it docs not know (except by indirect means), whether a packet for 
which no acknowledgment has been received was in fact processed by the intended node. 

When ordering is required, the challenge is greater Figure 2 shows commands sent in a 
particidar order A, B, C... by node A. and because of ringlet and transmission vagaries, reception 
in a different order, e.g. B, A, A, A,.... A well-behaved system should be able to reorder the sent 
packets and execute them properly, in the originally sent sequence. 

Figure 3 illustrates strong sequential ordering (SSO): no packet is received out of the €»-der 
in which it was sent — although it may appear twice, it does not appear out of order. Figure 4 
illustrates the use of both SSO and nonidempotent commands; not only is ^e sequence preserved, 
but because of the natiwc of nonidempotent commands, no command in a conventional system 
may be executed twice by the destination node. Note in Figure 4 that {produccrld, pLabel} arc 
common to send^kt and acknowledge; also, {producerld, pLabei} are visible to each node, 
absent errors. 

Figure 5 shows date structure suitable for applicant's solution to the above chaUenges; the 
packets include fields called produccrlD and pLabcl. whose details are discussed below. These 
two fields are included in the acknowledge packets sent back from the consumer to the producer. 

The present invention resends packets, both known good (already accepted/done) packets 
and others, in a pattern as shown in Figure 6. Figure 6A summarizes at a high level the basic 
approach of the system of the invention: a known good SSO packet Is resent, repeatedly if neces- 
sary. until a valid acknowledgment (which may be "done*' or "busy") is received at the producer 
node. Then the producer node resends all other packets after the known good packet, even those 
that have been properly processed, i.e. for which ack_done5 have been received. (See Figure 7.) 
As the acks come back from the resent packets, the producer node checks to see that all are valid 
acks ~ i.e. "done" or "busy". If they arc not all valid acks, then the resending process is repeated, 
until all acks are in fact valid. This ensures that all erroneous packets have been properly pro- 
cessed, and the system may then proceed with succeeding packets. Complicated challenges are 
presented in this approach to preserve SSO and support nonidempotent commands, and the solu- 
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tions are presented by the logic shown in Figures BASS, whose operation is illustrated in the state 
diagrams of Figures 39 et seq. 

The present invention is enabled by the fact that the consumer node continually maintains 
the state for packet actions (accept and reject stales) and for acks (done and busy); this goes for all 
packets and acks that each node sees. Thus, each consumer node is aware of the disposition for 
, every packet traveling die ringlet at the time that each packet passed through that node. This 
information is important in the rcscnd process. Moreover, each consumer node maintains a record 
of the sequence of packets that have gone through, and a sequence number of the last known good 
packet (for which an ack.done or ack.busy was issued). This provides a point to which the sys- 
tern can back up, if needed, when e.g. a CRC error is encountered. This will be more clearly seen 
in the discussion below of the example of Figures 39 et scq. 

Figure 8 is a block diagram showing a suitable hardware environment for the present 
invention, and serves also to show the possible flow of packets in the producer node. It can be 
understood in detail in light of the flow/sute diagrams of Figures 39 et scq. and the logic diagrams 
of Figures 8A-38. 

In Figure 39, the fundan^ental blocks of the system are shown. In the absence of the SSO 
ordering extensions, this system would include all blocks With the following deletions and 
changes: 

- Delete retry_pkt register, 7-190. 

- Delete retry_pkt queue, 7-30. 

- Delete orJast_ct counter inside 7-100; retain noJast.cL 

- Modify 4:1 mux to be 3:1 mux, 7-40. 

- Delete or_rcvd_ct counter, 7-250. 

Other blocks remain fundamental to tfie operation of a send node with no SSO support. 

The basic operation of the system is as follows, with reference to both Figure 8 and the 
more detailed block diagram of Figure 39. Packets arc inserted from the transaction layer into the 
link layer through the request and response FIFO queues. PackeU are selected through the multi- 
plexer (MUX) for (possible) transmission to the ringlet through the 5end_pkt register, the CRC 
generator and 8/20 encoder, and the serializer, to the ringlet, as illustrated. 

Acknowledges for request and response send packets from this node, as weU as incoming 
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request and response packets originating from other nodes but directed to this node, are received 
in the deserializer. After an 8/10 decoding and CRC cheeky incoming receive and acknowledlge 
packets are registered in the rec_pkt register. Idle symbols are striped before the rec_pkt regis- 
ter. CRC errors or 8/10 encoding errors invalidate any incoming receive and acknowledge packets. 
Acknowledges with CRC errors are not visible beyond the rec_pkt register, and received send 
packets from other nodes are processed so that they are not visible to the transaction layer, (See 
the receive buffer description.) The ack decode logic, detects acks addressed to this node for the 
send packets it had transmitted through its serializer. 

Acks are queued in the ack queue, and matched with send packets in the scndj>kt queue,, 
in the ack_logic bl€x:k. In the absence of SSO support, all packets would flow through the 
non.order register, with SSO, all SSO packet flow through the retry_pkt register, and non.SSO 
packets continue through the non- order register. 

The rcscnd logic basically examines ack status to determine whether a packet is completed 
(ack_done) or requires retry because of a busy receiver (ack_busy). Ack^donc packets return sta- 
tus to the transaction layer through the *ack to command level'* block. Non-SSO busy retry pack- 
ets may be selected to retry through the MUX logic and the 4:1 (or 3: 1, depending upon the 
selected embodiment of the invention) MUX. SSO busy retry packets are enqueued into the 
retry_pkt FIFO queue. They arc selected from this queue through the MUX logic, and the MUX, 
by the SSO busy retry logic, which is distributed throughout the system and described in the flow- 
charts for send logic, figures 9-25. 

The send logic of Figures 9-25 operates basically in the following manner. Its details of 
operations will become clearer below in connection with the example flow in relation to the state/ 
flow diagrams of Figures 39 et seq. Each of the logic units (e.g. ack.logic) in the following 
description refers to those elements in Figure 39. 

Figure 9: ack logic for testing packets against 

acknowledgments for error conditions, and starting busy retry 
in the absence of errors. 

Figure 10: ack logic for setting ack logic output for 

normal packet processing, in the absence of errors. 
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Figure 1 1: handling in the ack logic for non-SSO packets 
to be loaded into the non^order register. 

Figure 12: ack logic handling of the CRC_init_err error 

retry loop, which resends the last "known good" SSO packet until 
it receives a valid (done or busy) acknowledgment for this packet. 

Figure 13: ack logic handling the CRC^err error retry 

loop, which resends all packets following the "known good" packet 
back through the ringlet. 

Figure 14: ack logic handling the CRC_ack_^chk error retry 

loop, which tests whether all of the packets retried in the 
CRC_enr loop completed without enror or had an error. K any error 
is found, the loop is retried, beginning with CRC_init_err 
(fig 30). If no error is detected, CRC_crr_cnd is set. 

Figure 15: ack logic handling the CRC_ack_chk error retry 

loop, which tests the acknowledgment of each packet sent during 
the CRC_eiT loop (fig 31) for valid acknowledgment (ack.busy or 
ack^done) and sets an error bit if any error is detected. 

Figure 16: resend logic handling the fomiatting of packet 

state information and setting the MUX while executing the 
CRC_mit_err. CRC_err, and CRC_ack_chk loops. 

Figure 17: rcscnd logic to detect conditions to complete the 

CRC_crr_end control and set the busy_loop_cmplt control for 
detecting the completion of the busy retry loop, by testing for 
retry_pkt queue empty . 
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Figure 18: resend logic to detect SSO packets as done 

(ack.done) or requiring busy retry (ack_busy). Busy retry packets 
arc enqueued into the retry^kt queue; done packets arc signalled 
to the transaction layer. 

Figure 19: resend logic to detect non-SSO packets as done 

(ack_done) or requiring busy retry (ack_busy). Busy retry packets 
are sent through the MUX for retry; done packets are 
signalled to the transaction layer. 

Figxire 20: retry send logic to set the busy loop when no 

SSO packets are outstanding in the ringlety and to control the 
busy retry loop by setting the MUX to select packets 
from the retryjDkt queue through the retry send block. 

Figure 21: MUX logic to detect conditions for starting the 
busy retry loop, selecting a response packet 
from the response qucue^ or selecting a request packet 
from the request queue. 

Figxire 22: MUX logic to control the MUX to select 
a request from the request queue. 

Figure 23: MUX logic to control the MUX to select 
a response from the response queue. 

Figure 24: CRC generate logic to format controls and gate the 
transmission of packets to the serializer. 

Figure 25: special purpose modulo 64 comparison logic to perform 
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greater than (labeled .GT.) and less than (labeled UT.) con^arisons. 

What follows is a summary of the flow charts for the receive node, figures 28-38. Reference is 
made also to Ingure 26, Receive Node Logic 

To understand the extensions for this disclosure^ <me must first form an understanding of 
the underlying logic. In Fig 26. the fundamental blocks of the receive node in the absence of the 
SSO ordering extensions would include all blocks with the following deletions and changes: 

- Delete seqTable, 26-30. 

- Delete seq register, 26-40. 
« Delete seq update, 26-120. 

Other blocks remain fundamental to the operation of a receive node with no SSO 
support 

The receive node and send node have some conmuHi components with common 
nsmes. These include: 





send node 


revive no* 


deserializer 


7.240 


26.80 


mo decode/CRC check 


7.230 


26.70 


rec_pktreglstBr 


7.220 


26.60 


ack decode logic 


7.140 


26.140 


ack queue 


7.130 


26.150 



Non-SSQ Packet Plnw 

As alluded to above, the present system may be used in either non*$SO or fully SSO envi- 
ronments. 

A description of basic (non-SSO) packet flow, and an overview of the 
figures added for SSO support, follows. 
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Basic packet flow: non-SSQ packcte 

The role of the SSO ordering logic can be understood from comparison with the basic 
packet flow for non-SSO packets. Starting with Rgure 9. the blocks 9.10, 9.20, and 9.30 inhibit 
the nonnal packet handling if the CRC error handling control sequence of CRC_init_crr, 
CRC^cnr, and CRC_ack_chk are active. If all packets are non-SSO, then the block 9.50, which 
identifies the packet as non-SSO, sends packet control to figure 1 1.10. 

PigiuiB 11 describes the basic non-SSO error detection logic, comparing the head of the 
send^kt queue (see Figures 8 and 39) with the head of the ack queue to sec if the acknowledge 
matches the packet (Comparisons are done in blocks 11.10 through 11.40.) If the packet has an 
error, tfie block 1 1,50 sets the state *'err_ref* to signal this as an error packet Otherwise, the block 
11.60 forwards the "busy" and "done" states from the packet's acknowledge (at the head of tiic ack 
queue, 9.130). In both cases, the packet is written into the non^order.reg. 

In fig 9.10, describing resend logic for non-Ofdered packets, the non_ordcr^eg is tested 
for vaHdity. If valid, the packet is tested for "done" status in block 9,20. and "done" status is >MHi^ 
to the transaction layer in 9,50. If the packet is cither "busy" or has "cirjet" state set, then the 
4: ]_mux is set for resendS and the packet is forwarded to the send^kt rtgister, in block 9.G0. 
Finally, in this block the sequence count for non-SSO packets is incremented and assigned to this 
retried packet. 

In Figure 24, the scnd_pkt register is tested for validity in 24.10 and then forwanicd to 
CRC generation in block 2430. The pLabel field is defined by the fact that this is a non-SSO 
packet and by the assigned "seq" fidd in the same block 24.30. Also, the packet is again 
enqueued into the scnd_pkt queue to await an acknowledge packet 

While the flow charts begin with the arrival of packets, the packet is assumed to have been 
selected by some priority process. Jsx a non-SSO system, this priaity may be similar to thst of the 
system of the invention, but all references to "busy** and *'retry_pkt queue" wouU be removed. 
The selection of either a request or response packet firom respective queues in a non-SSO system 
would be similar to thst desoibed in fig 22 and 23. 

Logic additicm for SSO siippnrf 

The remaining ack logic, described in the flow charts 9-15, describes extensions for han- 



-10- 



(100) 



m^W- 10-164140 



dling SSO ordered blocks. 

(Note that the blocks 22.20, 13.20, 14.20» and 15^0 also identify non-SSO send com- 
mands in the send^kt queue (see Figure 39) and branch to the non-SSO packet handling block 
11.10. However, these blocks cover the handling of non-SSO packets within a system which has 
the capability of supportiiig a mixed programming environment of both SSO and non-SSO com- 
mands. That is. these blocks are not a part of a basic non-SSO only environment) 

Similarly, the handling of non-SSO packets in the resend logic block also is described in a 
single page. Rgure 9. The resend logic described hi Hgures 16-18 is added for SSO support. The 
retry handling logic, figure 20» is also unique to SSO iianHHtig logic. 

The MUX logic described in fig 21 to start the busy loop (set in 21.90) is unique to imple- 
mentations with SSO support Other logic, selecting request or response packets, would be similar 
for both SSO and non-SSO implementations. The MUX logic to initiate a request or response 
packet described in fig 22 and 23 would be similar for both SSO and non-SSO implementations. 

Finally, the CRC generate logic in Figure 24 tests the •no_xmit" sUte bit in 24.40 to deter- 
mine whether a packet is to be transferred to the scrialiadng logic. Such a gating is unique for SSO 
support The logic sui^rdng modulo 64 comparisons is also uniquely required for SSO support 

Fig 26 block diayram nfveryiew 

Fig 26, like Figures 8 and 39, is a potential block diagram of a receive node (send node for 
fig 7). used to illustrate how the concepts of the SSO mechanism may be implemented. It is rec- 
ognized that this is but one of many possible instantiadons of a receive (send) node embodying 
the concq)ts of this mechanism. 

Packet flow in Fig 26 is fundamentally as follows: 

- Rackets enter the receive node through the deserializer, 26.80, 
from the ringlet, 26P0. The CRC check and 8/10 decode logic. 
26.70, strips any non-packet symbols and loads packets into 
the 32 bit rcc_pkt register, 26.60. 

- The start of packet is detected and initial packet decode is 
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done in this register. Packets utilized by the receive node 
include: 

- send packets (request and response) addressed to diis node; 

• send packets (req and response) addressed to other nodes; 

- acknowledge packets addressed to other nodes. 

Send packets addressed to this node are marked for enqueing into 
the request.in queue (request packets) or response^in queue 
(response packets)^ respectively. (Note: the use of queues 

Acknowledges to this node are decoded and written to the ack 
queue. 26.150. 

- Receive node packets are written into the test j)kt node. When 
the start of the packet is in the testjjkt register, the 32 bits 
with the producerld arc in the rcc jpkt register. This field indexes 
the seqTable array for receive node state, including sequence the 
sequence and accq^t fields. 

- The next cycle, the seqTable[producerId] contents are read into the 
seq register. 26.40, and die pLabel and jn-oduoerld fields will be 

ill the tesLpkt register, 26.50. The corresponding fields are 
compared in the following blocks for all packets (send packets 
and acknowledges) in the receive node, although the tuning of packet 
handling depends on the type of packet 

• acpt logic, 26.130, basically determines whether request 
and lesponsB packets addressed to this node are accepted. 
Both the seqTable and CRC error checking are considered. 
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- seq update logic, 26.120, deteimines whether and how the 
seqlkble array state is updated at the end of the packet. 

- ack gen logic. 26.1 10» determines what kind (inhibited 
on CRC error detection) of acknowledgnDcnt is generated, 
ack done, busy, or error. 

- The remainder of the packet is processed (in the case of send 
packets), with any send packet dau enqueued into the 
request Jn (26« 10) or response.in (26.20) queues, prodded 
that queue space is available. At the conclusion of the packet, 
the CRC check completes the operadoa fcr the send node. 

Figure 27A and 27B.show receive packet and acknowledge timing diagrams, respectively. 
The receive node logic is shown in Figures 28 et seq.. and is described as follows. 

Figure 28: Detect the start of a send packet (or "command**) or an acknowledge packet from 

header fields in the rec_pkt_reg 26.60 (see Figure 26). For send packets, etdier requests or 
responses, addressed to this node, test whether queue space exists in the corresponding input 
queues, request_in queue 26.10 and response^i queue 26.20, and set control bits accord- 
ingly. 

Figure 29: Read the seq Table array 26.30 into the seq^reg 26.40. Test for condidons to inhibit 
validating a packet in the reqi]est.in 26.10 or response^in 26.20 queues. 

Figure 30: If control bits indicate queue space is available, tentatively enqueued the remainder of 
a packet addressed to this node in to the corresponding queue, request^in 26.10 or 
response_in 26.20. Conditions for incrementing control fickis or resettlDg control bits are 
given. 

Figure 31: At the end of a send packet addressed to this node, depending on the CRC check, gen- 
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erate an acknowledge packet to the send node and validate the packet in the request^ 26. 10 
or responseJoD 26.20 queue. 

Figure 32: Generate tcntatiYe acknowledge conditions (done, busy or error) for use in acknowl- 
edge generation on Figure 31, depending on CRC error checking. 

Figure 33: Coofcinuation of tentative acknofwledge generation (done, buzy, or error) for SSO pack- 
ets with valid sequence. 

Figure 34: Tbst for conditioas to tentatively set update conditions for the seq Table array at the 
conclusion of the packet, or to inhibit update, l^pdate wiU dq^end on valid CRC check. 

Figure 35: Ccmtinuation of tentative sequence update generation in the case wticre the packet is 
SSO ordered with valid sequence. 

Figure 36: At the conclusion of ttie send packet, test for valid CRC in 26.70. If the CRC is valid, 
and the seq Table write is not inhibited, update the seq Table. 

Figure 37: Generate seqTable update fields for acknowledge packets addressed to cnher nodes. 

Figure 38: Cmtimiation of seq Table update field generation for acknowledge packets addressed 
to otter nodes, for the case where packets are SSO ordered with valid sequence. 

The example of Hguies 39 et seq. demonstrate and exemplary operation of the system of 
the invention, with the state and acti(»as described in those Rguies, which confomi to the lo^ in 
the forcsKnng figures. Figures 39 et seq. show, step by step, the state of the producer node for each 
transfer of packets among the roisters and queues, and by inspection of those diagrams, it wiU be 
seen that the system does accommodate SSO» nonidempotent commands in the case of errors in 
connection with a busy retry loop and an ill-bdiaved node. 

In reading the logic/flow diagrams in the figures, various states are referred to. These are 
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defined below. In addition. Appendix A attached hereto includes a glossary of the tenninology 
and variables used in connection with the current design. 



Steifi Interpretation 

i in_proc 

b busy (variations: adc_busy, busy loop retxy) 

r retry in CRC loop 

bl first packet in busy retry loop 

rl first packet in CRC retry loop 

X either ack or busy 

d adcjdone 

BRC busy resend loop 

CRC CRC err resend loop 

SRL sendj>kt recirculating loop 



Discussion of the Preferred Embodiment of the Invention in Lig ht of Proposed 1394.2 Standard 
The following presents an overview and summary of a proposed mechanism to support 
fully pipelined packet initiation in a ringlet topology based on IEEE P1394^. where the pipelined 
packets may include send packets and response packets from address spaces lequiriQg strong 
sequential ordering (5SO) and support for non-idempotent commands. This mechanism can be 
extended over arbitrary PI 394.2 switch topologies by ensuring that every communication link in 
the topology, in tum» maintains SSO ordering. 

The proposed mechanism suppcrcs SSO and non-idempotent commands in a fiiUy pipe* 
lined mode of operation for non-busy, non-error packet transmission, while tolerating both CRC 
errors and busy receiver conditions through an efficient hardware retry mechanism. It supports a 
variety of resend policies for mixing new packets with retried busy packets. It enables a **correct- 
able error" kind of error handling policy, ance CRC error packets may be retried to any prpp-am- 
mable limit This capability, in turn, enables a "preventative maintenance'* function for nodes 
reporting successful retry operations of CRC error packets. 

Also note that the use of SSO, non-idempotent capability within a bridge significantly 
simplifies the design of bridges between IEEE P1394.2 and other protocols that have larger (>64 
B) packet fomiats. For example, a write command for a 2 KB block from an IEEE 1394-1995 
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node, which expects a single response packet from the receivuig node at its conclusioD, can be 
implemented as a series of move commands (no response) and a concluding write, with the guar- 
antee that the operation of the IEEE 1394-1995 packet will be preserved over the bridge. 

The cost of SSO oidcnng is 2 bytes of state per node supported on the ringlet, plus state 
machinet, a separate register for non-SSO packets, muxes, and a 6 bit comparator. The How 
description is now based on simple FIFO queues and registers, rather than more complicated array 
structures. Since the maximum configuration for a ringlet is 63 nodes, this bounds the required 
state to support SSO to -128 B max (plus pointers and state machines). "PrafHes" of smaller sup- 
ported configurations can further reduce the cost by cutting the supp(Kted ringlet node count. 

Brief OYcrvigy of IEEE F 1394,2 

IEEE P1394.2, or Serial Express, is a proposed extension of IEEE 1394-1995» or Serial 
Bus, to Gigabits- transmission levels. BasicaUy, the protocol supports the fundamental command 
set of IEEE 1394-1995 while rearchitecting the transfer protocol to emphasize low latency and 
high tfarougfaput with 16 B and 64 B packets. 

IEEE P1394^ is based on an inseition buffer architecture with a riiig^et topology. What 
this means Is that a each node includes two unidirectional wires, one for packets and idles (for 
synchronization and flow control) coming in, and one for packets and idles going out. A bypass 
buffer shunts packets and idles coming in over to the outgoing wire, possibly with delays. A sin- 
gle new send or response packet (defined below) may be inserted by the node into the ringlet if its 
bypass buffer is not full; this is the "insertion buffer** architecture. Any packet that enters the 
bypass buffer while the node is inserting the new packet is delayed until the new packet is sent 
The BmaU packet sizes (<C= 64B) make this archibectuie both possible and effi«ent. 

The packet fomiat includes a 16 bit field for packet routing usage plus a 4S bit extended 
address. Each node supports full duplex operation over a single cable with separate send and 
receive paths. Whm several nodes (up to 63) are interconnected, initializadon software breaks 
redundant loops and configures those nodes as a single ringlet. Multiple ringlets (with as few as 2 
connections) can be interconnected through switches, up to a maximum topology of 16X nodes. 

The IEEE P1394.2 protocol supports four modes of operation, based on two address 
modes (directed and multicast) and two service types (asynchronous and isochronous). The SSO 
ordering is of particular interest for asynchronous, directed service, which includes (read, write. 
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move, and lock) operations, but the proposed mechanism is not limited to only this mode of ser> 
vice. 

SSO Ordering: Issye^ 

In the absence of SSO ordering support, packets from any node to any destination have no 
constraints with respect to one another. With both CRC errors and busy receivers to contend with, 
this means that any two packets may arrive at the destination node in a different order from that 
sent, if at all. 

This uncertainty is typical of networked environments and is dealt with using a variety of 
software-based protocols. One reliable mechanism, for example, that can be used with Trfp 
1394- 1995 to serialize an interrupt generating write request with a prior data packet, is to send a 
write command (which requires a response from the destination) for a large, say 2KB packet, and 
then send the write command to generate th^ interrupt after Ike response for the data has come 
back. 

Such mechanisms, when extended into a domain with much smeller (<» 64B) packets, a 
domain that emphasizes low latency transfers, can become both complicated and inefficient The 
software overhead for establishing ordering and data transfer coo^ledon both detracts ftora avail- 
able compute cycles and adds directly to user process - to * user process latency. 

The SSO extcnrirm with Pl^^d l 

The proposed SSO extension depends on these fundamental assumpticnis for the 
operation of a ringlet: 

Each command in (asynchronous, directed) mode is executed by the originating (sender) 
node generating a send packet and, depending on the command, the ultimate destination node 
(possibly) generating a response packet The "send" packet is removed by a single receiving node 
on the sender's local ringlet and replaced with an "acknowledge" packet from that local lecciving 
node back to the sending node. The local receiving node may be either the final destination for the 
send packet or an agent that forwards the packet, perhaps through a switch, to its destination, 
Depeading on the type of command, the ultimate destination node may send a "response** packet, 
which is also captured by some node on its local ringlet and replaced by an ''acknowledge** packet 
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back to the "respcuise" generator. 

In the dificusfiion below; the terni "producer node'* is sometixnes used to indicate the node 
that initiates the packet, whether that node is a sender node initiating a send packet, a receiver 
node initiatijag a response packet, or abridge or switch node forwarding a send or response 
packet 



The target for both send and receive packets is identified by a global, 16 bit "targetld** 
field. In the absence of errors, some unique node on the ringlet, perhq>s a bridge or switch node» 
will recognize the tai^getld address and strip the packet. A "sourceld** field uniquely Identifies the 
global address of the sending node. 

Other fields, which are included within the packet for returning local ringlet acknowledg- 
ment, are fimdamental to the proposed SSO ordering enhancement These include the (6 bit) *'pro- 
duoerld** field, which identifies the nodeld that originated this packet '*'*'an its local ringlet^**, and 
the (8 bit) "^pLabel** field, which is assigned by the "producerld** node to uniquely Identify this 
padcet B oth the producerld and pLabel fields have meaning only within a local ringlet and will be 
reassigned, for example, if the packet is forwanied through a switch. Similaity, the ^'producerld" 
and ''pLabel'* fielda are also reassigned by die destination node when it sends a response packet 

Other fields mentioned below include the "type" field (which distinguishes types of com- 
mands and acknowledgments); the "code** field (command identification); the global "tLabel" 
fidd, which is returned in a response packet to uniquely idend^ that response to the sourceld 
node; and the crc32 field, a 32 bit CRC code that covers both the header and data for the packet 

{transmitted first) 

3 2 1 0 0 

2 4 6 8 0 

I I I I I 1 1 I 1 t I t I I I I I I [ I I I t i t I I 1 I I I I I 



I target td | I type I I code \ 

i tLabel t I pLabel t I producer Id I 

I sourceld | of f aetai (addr) | 
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I o££setLo(addr) I 

( optional data ) 

I crc32 I 



The acknowledgznejit packet generated by the receiver node echoes some of the fields 
from the send or response packet back to the ''producer node", so that it can uniquely recognize 
the packet bdng acknowledged: 

(transmitted first) 

3 2 10 0 

2 4 6 8 0 

I I I I } 1 t I I ) I I 1 t I I I t .1 t I I I I I 1 I I ( I I I t 

I localBus I producer Id i I type I \ pLabel | 

I crc32 i 

To create the acknowledge address, the "taigcOd" address field of the original packet is 
replaced by a "localfius** identifier (all I's, typically], plus the 6 bit pr oducerld field from orig- 
inal packet. The pLabeL field uniquely idcndfies the original packet to the **prDduDer node**. The 
"type" field encodes the type of acknowledgment. The crc32 field for the acknowledge packet 
may be replaced by some other fsrror detection code without loss of generality for the SSO mech- 
anisTii described here. 

Fundamental to the proposal is the assumption that local rii^et transmission is unidirec- 
tional and bypasses no nodes. That is, when a node sends either a send packet or a response 
packet, then every node on the subring will see either that packet or the acknowledgment for that 
packet flow through the node's bypass buffer. 

Tkus assumption underlies the basic operaticxi of the P1394.2 ringlet (although in a pro- 
posed P1394.2 option, a "short cut** routing feature would not support SSO ordering). As a result 
of this, for every send response packet that is transmitted, each node on the ringlet can observe 
both the producerld and the pLabel fields for tithtt the senci/response packet or its acknowledg- 

•1^ 



(109) 



^mW- 10-164140 



ment. 

The PI 394^ protocol suppom split response transactions, where a response is required. 
Padcets for (asynchronous, directed) service are subject to two conditions that may inhibit packet 
delivery: 

the packet - or its acknowledgment • may suffer a CRC 

error or other detectable error; 
the packet may be rejected by a *1>usy** receiver at 

the destination node. 

In the case of a CRC error, the ^y reliable detection is at the originating node, which will 
detect the error if the expected acknowledgment has not been seen within two ringlet revolutions. 
(A bit is circulated by a "scrubber" node that is ccrapkmcnted at every ringlet revolution to assist 
in local node timeout detection.) The exact method of timeout detectiM employed is not funda- 
mental to the (^>eration of the SSO ordering support mechanism described here. 

In the case of a "busy" receiver, the receiver node substitutes an ACK.busy acknowledg- 
ment packet for the send or response packet in (asynchronous, directed) service. 

System overview: known good information at a node 

The claim that the pLabel and producerld fields can be used to implement an SSO order- 
ing mechanism in a ringlet topology depends on certain information being unambiguously known 
to both the producer node and the receiver node. Here is a summary of this infomiation. 

Hie basic concept here is that each packet may have two uses in the ringlet: to tnumnit 
commands or data in the SSO program space, and to transfer ringfet seqaence state information. 
Only the sender knows the sequence number of transmitted packets* and only the receiver knows 
the last valid, acc^ted packet. Pictorially: 

+ Sender < + 

I t 
Error on send pkt?? ? Error on ACK? 

I I. 

+- >Receiver + 
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Produccx nodes: 

- SeDd/respoxise packet fields to control SSO operation in ringlet nodes: The pLebel field 
for send acid response packets from each producerld is used to convey three subfields (details 
later): 

pLabeLsso (1 bit) 

pLabel.bzseq CI bit) Unused for non-SSO space V 
pLabeLseq (6 bits) 

The "pLabeLsso** bit defines, dynamically, whether the address space for this packet is 
SSO ^dered (sso » 1) or not (sso =» 0). The "bzseq". or busy sequence bit has meaning only an 
SSO ordering space and is used to help a receiver node determine whether it must issue a busy 
retry acknowledgment to a send / response packet. The 6 bit "scq" field is a wr^ping. modulo 64 
''counter" used by all local ringlet nodes for identifying valid packet sequences from this produc- 
erld. 

The maintenance of SSO ordering between eadi producerld node and the other nodes on 
the ringlet depends on the producerld ''metering" its send and receive packete and setting their 
pLabel fields to ensure that each other node can maintain proper state informatian. By comparing 
that state information with incoming pLabel fields from the producerld, packets can be properly 
dispositioned to maintain SSO ordering. 

("Metering" requires that any producer node may have no more than 32 packets ''outstand- 
ing*' in its local ringlet, based on the 6 bit "seq" field. "Outstanding** means counting from the cur- 
rent packet to send back to either the earliest prior packet linom this node that generated a "busy" 
acknowledgment, ot the earliest prior packet for which no acknowledgment has been received, 
whichever is greater. There is no architected limit to the number of packets that may be outstand- 
ing beyond the local ringlet, for example awaiting responses at destination nodes.) 

Under normal operation in SSO space, the pLabel.seq field vs incremented by 1, modulo 
64. with each new scndAiesponsc packet or with each retried busy packet. If a CRC error is 
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detected, the SSO state for all local rin^et nodes must first be resyncbed by resendicg some pack- 
ets (below). 

The term *'CRC eiror** used below may be generally understood to include not only CRC 
errors, but any noa-corrccted error condition for a packet (send, response, or acknowledge) that 
can be detected by the sending node. 

CRC error loop reset: Once a CRC error is detected at the producer node, the SSO state 
values nudntained for this producerld at every node in the ringlet are in question. Each ringlet 
node's SSO state informatioa for this producerlD must be reset, without any change in either the 
SSO programming space for this producerld or for any other producer node. 

Two importani features of the mechanism are used to accomplish this: 

1 . Each receiver node on the ringlet maintains an array of SSO state, indexed by the 
local producerld for each sending node on the local ringlet. The SSO state contains 
control bits plus two sequencing fields. One of these fields is used to determine 
whether a current ringlet packet addressed to this receiver has a valid SSO sequence 
number, and the other field is used to determine, along with the state of the node's 
receive buffer, whether the packet may be aoc^ted or rejected by the node and what 
type of acknowledgment will be returned (done, busy, or etror). 

2. Bach producerld that detects a CRC error has the ability to retry a sequence of SSO 
ordered packets, beginning with the last gtxid SSO ordered packet prior to the error 
packet, and extending through the last packet sent from this node prior to the detec- 
tion of the enror. The known good SSO ordered packet is retried (if necessary) to 
begin the CRC error loop, in order to establish a valid sequence base count in the 
ringlet. Then the remaining packets are resoil. up through the last packet t>efare the 
CRC error. If any wor is detected in this retry sequence, the loop is rq>eated until 
either it succeeds without error, or the error retry loop concludes without suocess; in 
such cases, higher level protocols may deal with the error. Once this sequence com- 
pletes without error, all nodes in the ringlet will have their SSO state restored to con- 
sistent, known good values . 

Receiver nodes: 
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A receiver node can detect a valid packet sequence from any produoerld if its SSO state infonxm- 
tion is correct This state is initialized b> the produccrld on reset and then re-initifllized if a CRC 
erm* is detected by the producerlD. 

A receiving ncde may or may not detect a CRC error send packet or acknowJedge. Even if 
it does, since the packets contents are unreliable, no node state information can be updated. It is 
the sending node's responsibility to detect a timeout and res cad any CRC error packcL 

Hmrever, the receiving node can detect an out<of-sequence transmission of a valid 
ordered packet following a CRC error packet from some producerlD. Each such out-of-sequence 
packet must be detected and rejected at the receiver while being acknowledged as ACK^eiror. Tbe 
node sending this packet marks the packet and retries it in its error loop. 

A packet may be rejected in asynchronous operation if the receiving node is busy or, with 
SSO catering support, if the packet is a retransmission of a packet that had been previously 
accepted. In SSO ordered mode, once a first packet Jrom a given producerld has l>©en rejected as 
busy, then all subsequent ordered-packeta from this producerlD must also be rejected as busy until 
the first busy packet is retried. 

The state of the receiver for SSO must then support the acceptance of new packets into a 
non-busy receiver buffer and the rejection of previously accepted packets that are resent for error 
recovery and any new packet sent following a "busy" rejection. Since the acknowledge packet 
returned to the producer is subject to error, the receiver sute must be able to reproduce the 
ack^done and acl;_busy packets if any packet is retried. 

To accomplish this, each recaver node requires a state array, called seqTable, indexed by 
the producerld* with the fdlowing six ilelds: 

seqTable. verif (producerld] (1 bit) 
seqTable. SSO [producerld]' (1 bit) 
seqTable. seq [producerld] {6 bits) 
seqTable. bzseq [producerld] (1 bit) 
seqTable. busy (producerld] (1 bit) 
seqTable. acpt [producerld] (6 bits) 
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Li SSO operation, the teqTable fields are used as follows. The "verif ' field indicates botfi 
that this seqTkble has been initialized and that the produoerld is a verified source. The ""sso" field 
defines the (static) capability of this produoerld node to support SSO ordering. (By comparison^ 
the pLabeLsso bit is a dynamic bit set according to the address space ordering for the producer 
node.) The "seq" field is compared with the pLabel "seq'* field for packet sequeocing by the 
receiver node. The "bzseq" bit is compared with the corresponding pLabel "bzseq** bit for SSO 
packets to identify the retransmission of first busy packets. The "busy" bit is set on the first busy 
acknowledgment of an SSO packet to the produoerld and (possibly) reset when this busy packet is 
retried, depending on the receiver bufifer. Finally, the "acpf* field is used in both the acceptance 
and rejection of SSO packets and in determining whether to acknowledge as **done** or "busy". 
These determinations require comparison with the incoming packers "seq" field. 

The receiver node can detect a valid new, SS0 ordered packet from a given producerlD if; 

pLabel.seq » seqralDle. seq[ produoerld] -h i 
If the comparison is valid, the seq field is updated: 

seqTable.seq [produce rid] - pLabel. seq 

For other comparisons, see below« 

The scqTablcseqlpnoducerld] field b reset to the cuirent pLabel.seq value when this node detects 
that this produoerld is iririHllTing a CRC eiror retry loop. This is indicated by con^Muing the seq 
fields for a packet firom - or acknowledge to - this ^roducerldb 

if {pLabel.seq <*= seqTable . seq [produoerld] } then 
seqTable. seq (producer Id] = pLabel.seq 

Only the seqlkblcseq value is modified, not the seqT&ble.acptQ field, during the initiaUza* 
tion of the CRC error loop. The packet will be rejected, but using the same acceptAeject logic nor- 
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mally used (below). It iz m^>ortam thAt the seqTabLe state infbnnation be used to ocrrectly 
acknowledge all icciied padocts as 'done" (meanipg previously accepted) or ''busy"; the logic is 
described below. 

If the "seq- comparison is valid, then packets are accepted or rejected according to this 
comparison: 



accept: { {pLabel .bzseq seqTatjle .bzseq [producerld] ) 
& { receive r^not^busy} 
I {pLabel. bzseq — seqTable.bzseqrproducerldJ } 
& {seqxable.busy [producerld] = 0} 

5 (receiver_not_busy ) 

6 (pLabel, seq > seqTable.acpt [producerld] } J 

reject: { {receiver_busy} 

I (pLabel. bzseq = seqTable. bzseq [producerld] ) 

& (seqTable. busy [producerld] « 1} 
I (pLabel .bzseq - seqTable .bzseq [producerld] > 

& (pLabel. seq <- seqTable.acpt [producerld] ) } 

« accept 

Less f onnally, packets are eligible for acceptance if a busy retry loop is being started and 
the receiver buifer is not busy, or if a busy retry loop is not being started and the "busy* state is 
reset and the receiver buffer is not busy and the sequence canq>uison shows that the producer 
node is not in an error retry loop. Packets are rejected if not accepted. 

If the •'seq" comparison is valid, then packets are acknowledged "done" or '^busy** accord- 
ing to this comparison: 

ack_don; ( (pLabel .bzseq seqT able, bzseq [producer I d] } 
6 ( receiver_not_busy) 
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1 {seqTable.busy [producerld] » 0) 

& { receive r_not_busy ) 
I {pLabel.seq < seqTable . acpt [producerld] ) } 

ackjbzy: { {pLabel . bzseq • seqTable, bzseq [producerld] } 
& {seqTable.busy [producerld] - 1] 
& {pLabel.aeq >- seqrable -acpt [producerld) } 
I {pLabel.seq >« seqTable. acpt [producerld] ) 

& { receive r_busy> 
• - ack_don 

Less fomudly, packets are seat ack.don if a busy retry loop is being started and the 
receiver is act busy, or if the "busy" state is reset and the receiver is not busy, or if the sequence 
comparison shows that in an eiror retry loop, the packet being retried had been sent before the first 
"busy** packet. Packets are acknowledged as acl;_bzy if they arc not acknowledged as ack^don 
Cagain with the assumption of valid sequence comparison). 

" The first time that a receiver node reject^ a packet as busy from a given producerld, it 
must change its "busy** state by setting (seqTable.busy [producerld] ~ I). This has the affect of 
freezing the seqTab]e.acptQ field at the *'busy** packet's sequence number. This field ronains fro- 
zen unti] the first busy packet is retried. 

Conditicns for setting "biisy" and "aq)!** fields depend on whether ia busy retry loop is 
being executed by the producer node. If the "seq** comparison is valid, then the first packet in a 
busy retry loop can be detected from 

pLabel.bzseq seqTable »bzseq [producerld] 

(Note that the "bzseq" field is not changed while executing an error retry loop. Since this is true, it 
will always be true that 

{pLabel . seq > seqTable . acpt [producerld] ] 
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in die packets of a busy retry loop.) 

If a first retry packet is detected, then seqlUble entries are set a f oUoWs: 

{ seqTable,b2seq[producerlclJ = pLabel .t)zseq 
3eqT able. busy (producer Id) - receiver__bu5y 
seqTable.acpt [producerld) « pLabel.seq ) 

Note that the first retry packet will be rejected by the compaiison criteria above and 
acknowledged as either "done" of "busy". If a packet is not a first retry packet, this is detected 
from 

pLabel«bz8eq « seqTa^le.bzseqlproducerld] 
Then seqTable entriea depend on packet acc^tance: 



if { (seqTable.buay [producerld] - 0} 

& {pLabel.seq > seqTabie.acpt: (producerld] > ) then 
{seqTable.budy {producerld] - receiver Jbuey 
seqTable. acpt (producerld] - pLabel . seq 

{ ■ 

else /* previous busy packet or error retry loop*/ 
{no update 
} 

Finally, each node can detect a packet with a sequence error from the producerld if: 

pLabel.seq > seqTable. seq [producerld] + 1 

This can happen if this packet has valid CRC but some prior i»cket did not. (The count 
oomparisons are not full 6 bit arithmetic can^arisons; see details below.) No seqTable entries are 
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updated for packets with sequence eirors. 
Ringlet SSO state inidalizatioa: 

- The producer node must ensure that the state information at each node on its ringlet is 
initialized before sending actual SSO send^esponse packets. 

A "trusted producer" politqr may be supported with the proposed scqTable state l>its and 
write send packets to initialize the seqTable. Thrae steps are needed: 

1. TheseqTableO airayispovn^onorresetwithseq7^ble.vcnfn aOfora^ 
Grid's. 

2. Through an (unspecified) process in a non-^SSO address space, the local node verifies a 
proposed producerld and then writes its own seqTable. veriflproducerld] = 1, The 
-verif ' bit may only be written by the local node. 

3. With seq7kble.verif[produccrI4] activated, the producerld sends a write (or move) 
command add r e s sed to this node's seqTable. The packet is identified as an inidaliza- 
tion packet through its target address Cseql^ble). 

The values written into (be seqTkble are taken from the initializatioD packet's data as fol- 
lows: 

Lt { seqTable. verif [producerld] ~ 1) then 
{ seqTable «s3o (producerld] - 1 
seqTable. seq [producer Id) = [current "seq" 
from this producerld) 

seqTable. bzseq [producerld] « [current "bzsGq" 

from this producerld] 
seqTable. busy (producerld] = 0 

seqTable. acpt (producerld] » [current "seq" 

from this producerld] 
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1 

Each node in &e ringlet must be verified and initialized separately. 

A policy like the above can be utilized to dynamically add (or subtract) nodes from an 
existing ring as well as initialize a newly powo^-on ring. 

A policy like the above can enable sclectivB "trusted node" exchanges. For example, node 
A can exchange with node B , node C with node B . but A and B can be prevented from S SO 
exchanges. 

Application tpAOft 

The proposed mechanism can operate in a generalized 1394.2 network topology 
which includes: 

nodes that are SSO-oidcring capable and those that are not (static capability); and 
nodes that support multiple address spaces that may be cither SSO ordered or not SSO 
ordered (dynamic capetnlity). 

Note that dynamic capability is important since within IEEE PI394.2 there are modes 
(isochronous) that may not be SSO ordered, and bridging to non-SSO ordered spaces (such as 
IEEE 1394^1995) must be suppated. GcncraUy, it is presumed (not required) that the software 
interface to the IEEE PI 394. 2 node is a set of address spaces with potentially varying program- 
ming models. At least one of these models is presumed to be an SSO model supporting non-idem- 
potent commands. 

The applicatim of the SSO ordering mecdianism is described here as it applies to the tphr 
P1394^ (asynchn»ou8. directed) service. This does not mean that it cannot apply to other modes 
as well. This mechanism supports an arbitrary stream of commands from mult^dc addness spaces, 
meaning that SSO commands may be interspersed with commands from ottier programming mod- 
els. 

If SSO ordering is to be maintained, end-to-end, from an SSO address space (or SSO 
domain) to a destination node, then it is clear that the destination node for SSO domain com- 
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mands must be SSOmlermg capable, and that all intennediale nodes in switches or bridges must 
be SSO-ardering capable. 

TEHP. ?nQ4.2 Changes requiied for supp cHi 

The proposed SSO support mechanism can be implemented with the current 0.6 version of 
the spec without modification as an undocumeated optional feature. When patents covering this 
mechanism axe filed, however, it is planned that this mechanisra will be added to die n?FP 
P1394 J docianentadon as (at least) an optional feature. 

While there is no required change, one encoding addition is suggested for clarity and 
another encoding addition is needed for support of cost-effective bridging to twpw 1394.1995 
nodes. 

The encoding addition for clarity is the addition of an " ACK^crror" acknowledlgfnem to 
the "type" iield. replacing a reserved code: 

TVpefiektO Normal, non-SSO 

1 Extended, non-SSO 
2 [reserved] 

3 [reserved) 

4 ACK_done 

5 ACK_jDusy 

6 ACK_more /* Multicast mode retry req'd 
*7ACK_errQr*/* Sequence error in SSO mode */ 

The encoding addition for bridging is qiecifying bits to delineate the starting, middle, and 
ending 64 B packets whose destination is reached through an interconnect that supports longer 
packet types. For exaxnplc. 2KB packets over Bthcmct or 1394 can be broken up into three types 
of packets: 

Start packet: possibly a partial block, aligned at its end on a 64B boundary. 
Middle packet: always 64B, always aligned. 

End packet: possibly a partial block, aligned at its start on a 64B boundary. 
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A kxge packet entering a P 1394«2 ringlet through a bridge may be broken up into smaller 
packets (for cxainplc. a large write packet broken into multiple 64B moves plus a 64B write, for 
example). But reassembling a large packet ^om even SSO ordered 64B packets, for example to 
bridge to a 1394 destination node, requires that the component parts of the large packet be easily 
identified. The need is particularly strong because of the typically low inefficiency of protocols 
designed for large packets when dealing with small 64B packets. A start/middlc/cnd encodings 
when combined with the SSO ordering capability, allow a bridge to very efficiently reassemble 
packets for delivery. 

Retry overview 

1^0 types of retry conditions must be dealt with for SSO ordered addiess spaces support* 
ing non-idempotent commands: "busy** conditions at the receiving node and CRC error conditions 
detected in the local ringlet. In the latter case, it is assumed that **CRC error** means the corruption 
of the packet (either the send packet, the response packet, or the ACK to either) so that neither 
source nor destination fields represent reliable data. The term "CRC enor" should not be con- 
strued to mean restriction to only CRC error codes, but more generally ahoidd include any error 
detecting mechanism. For efficient operation of ordered domains, b assumed that "busy** letiy 
conditions are relatively infrequent but still much more irequent than CRC error conditions. How- 
ever, in no case is the validity of the proposed scheme dqacndent on the frequency of occurrence 
of cither "busy" or CRC error cases. 

For *'busy*' retries in ordered doorudns, in the absence of errors, it is only necessary to retry 
packets to those specific domains which respond "busy**. The proposed scheme treats **busy** con- 
ditions in alternate destination nodes as independent; if node A transfers a sequence of packets to 
both nodes B and C» and only B responds "busy", then only the packets to node B am xetried. 

CRC error packets (either send/respmse packets or ACKs) are ultimately detected at the 
sending node. In SSO ordered domains » this detection might be cither as a "sequence errcH''' (an 
ACK^err response is detected before a time out) or as a ringlet time out (two revolutions with no 
acknowledgment). In SSO ordered domains, receiver nodes have the responsibility of sending an 
ACI^error to reject each send/response packet foliowing a CRC error occuxrence if a sequence 
error is detected from some node, but otherwise must maintain no error state information. When 
the sending node does detect that a CRC error has occurred in either its packet or an ACK, then it 
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must retry all outstanding S50 ordered domain packets from the point of the last valid, completed 
packet up through the last outstanding packet sent before the error Is detected and packet trans- 
mission is halted. The policy for reb^iiig CRC errors in non-SSO domains is imspecafied ho-e. 

Busy retry example 

The proposed mechanism for SSO ordering support is illustrated with an example before 
detailed descriptions are given, in order to better visualize how the mechanism would work. 
1 

Case 1 : Busy retry hrom node A to node C: 

Packet sequencing fnvn standpoint of the send packet: 



Node A aend values i 
Destination : B I 

pLabel.seq: 20 

pLabel.bzseq 1 

Busy retry: 

Node A rf^rPivP va1np«g 



C 
21 
1 

21 
I 



B I 

22 
1 



C 

23 

1 

23 
I 



B 
24 
0 

24 



C 
25 
D 

25 



I < bzy loop > I 



Busy detect: ...21 bzy 

Ngde B Yflluea; 

Packet conqpletlon: 

Done Done 
Accept/re ject : 

Acpt Aqpt 

seqrable . eeq [producer id] : 

19 20 21 

seqxable .bzaeq [produceriO] : 

1 11 
seqrable .busy Cproducerld] : 

0 0 0 

seqrable .acpt [producerldj : 

19 20 21 



.23 bay— ^ 



22 
1 
0 

22 



23 
1 
0 

23 



24 
0 
0 

24 



Node C values- 
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PbcIdcI oompIeitaQi 

Acoept/raject 

seqrnbl&seqlpraducerld]: 
19 

6eqTable.bz8eq[producarId]: 
I 

secfTabkiausylpfodiiccridl: 
0 

19 



Bzy 



20 



20 



21 
1 
I 

21 



Bzy 
Rej 



22 
1 
1 

21 



Done JXbb 
Aqit AcpC 



23 24 
1 0 

1 0 

21 24 



Some observations about this example: 

Packets are generated through the send port, but the busy acknowledgment is detected 
asynchronously in the receive port. The busy retry Icxjp requires that all outstanding 
prior transactions be acknowledged prior to retransmitting the firxst retried busy packet 
(24). Also, the decision on when to retry busy packets may depend on other implemen- 
tation cansideratioxis» such as the arrival rate of new transactions. In this example, 
packet 23 is the last packet sent before the busy loop lescnds packets 21 and 23 as 
renumbered (pLabdjseq) packets 24 and 25. 

Only packets marked busy (in feis case, only those to node beginning with packet 2 1) 
arc retried; there is no need to retry completed packets. 

New se^juence numbers are assigned fay node A to the retried packets. 

Once a first 1)usy- packet is acknowledged at node C (packet 21), then every packet with 
the same or higher sequence number addressed to node C (packet 23 in this example) 
must be rcjec^tcd with "adc.busy " until the first "bus/* packet is retried (inHicatyd by a 
change in "bzseq'*). In an error retry loop, it is possible that a packet prior to 21 is 
retransmitted. In this case, the packet would be rejected and acknowledged as 
•'ack^done'* to prevent its being (incorrectly) retried. Every other node that acknowl- 
edges a first "busy" must behave the same way (not shown in (his example). 

SSO ordering is maintained, pairwise, from the sending node to each of its potential 
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receiving nodes. However, if node A send SSO ordered packets to node B. and A also 
sends SSO ordered packets to node C it does '*'*not'*^ foUow that tbe relative ordering 
packet arrival between nodes B and C must be maintained for paiiwise SSO conect- 
ness. 

The beginning of a busy retry loop by this produccrld node is signalled to all ringlet nodes 
by complementing the pLabcl.bzseq field (from 1 to 0). This swap can be detected at 
each ringlet node by cwnparing its scqTable.bzscqfproduccrld] value with the incan- 
ing pLabel.bzseq; if they disagree, then the seqTable value is rewritten: 

seqTaJ3le.bzseq[proclucerId] « pLabel.bzseq. 
Also, if the retried busy packet is accepted in the receive buffer, then the "busy* bit is teset: 

seqTable. busy (producerldj - 0. 
For all other (non-receiver) nodes, seq'Dible.busyEprodiicerld] is set = 0. 

CRC enror retry examples 

Two examples of CRC enror retry without busy acknowledgments are followed 
with an example of a CRC eiior occurring around a busy retry loop. 

Case 1; CRC enror on send packet to node C, detected at node C. 

Packet sequencing ^om standpoint of die send packet: 
Case 1: CRC error on .spnri T^arirAi- fr^ ppd^. p. rfo h ^r^^ri nod^ c. 
Packet sequencing from standpoint of the send packet: 
Node A send values: 



Destination: 


B t 


C 1 


B 1 


C 1 


B ( 


C 1 


B 1 


C 


pLabeLscQT 
* 


20 


21 


22 


23 • 


20 


21 


22 


23 


pLabeI.bz8eq 


1 


1 


1 


1 • 
• 


1 


1 


1 


1 


OElCeiTietiy: 


20 








20 
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.1 ^ 



Node A receive values' 








-loop- 


>| 




Buy detect 












Node B values: 














Paclcelcosnpletioa: 




* 










Done * 


Dooe 




Done 




Done 






Acpi 








Rej 




seqT«ble.icq^radiioerld]: 




• 










19 20 


21 


22 • 


23 


20 


21 


22 


8eqTabIe.bz8eq[prcxiiioerId]: 




• 






1 ] 


1 


1 ♦ 


1 


1 


1 


1 


8eqTable,busy[prodticerId]: 




♦ 










0 0 


0 




0 


0 


0 


0 


9eqTable.aqrt|procluoerld]: * 














19 20 


21 


22 • 


23 


20 


21 


22 


Node C values: 














Packet ccOTplPition: 














(CRQ 




Eir • 








Dooe 


AcoeptAeject: (enr) 








Acpl 




Acpt 


seqTabk^seqlproduoexItQ: 














19 20 


20 


20 • 


20 


20 


21 


22 


seqTableinMqIiptodimrld]: 








1 1 


1 


1 • 


1 


1 


1 


1 


seqTaUe.biisy^sroduoBdid(]: 












0 0 


0 


0 * 


0 


0 


0 


0 


6eqTablB.acpt(piodiioerI4): 














19 20 


20 


20 * 


20 


20 


21 


22 



Some notes on this eauux^le: 

The detection of a CRC error occurs within the receive node» asynchronously from the 
send node. lii this example, packets 22 and 23 are transmitted before the CRC error for 
21 is detected. 

Because of the many locations that a CRC error can occur, different nodes may detect - or 
fail to detect the error in dififerent ways. In this example, the CRC error occurs aftcr 
nodc B, so it never sees any sequence enror. Node C, on the other hand, fails to detect 
packet 21 and then detects that subsequent packcts/acks 22 and 23 have sequence 
errors, since its last reconled sequence value is 20. 
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Once a CRC error is detected at the sender, its CRC error loop must fir»t retnnsxmt the last 
good packet before the error (packet 20). repeatedly if necessary, to reestablish the 
sequence couct (to 20} within the 6eqTahle.8eq[producerId] field of each local ringlet 
node. Based cm camparison with the receiver node's scqTable.acpitproduccria] field, 
this packet will always be rejected, regardless of the target. Along with djc sequence 
number for the known good packet, the prodncerld also sends out the ctinent bzseq 
value in pLabel to set to a known good state every seqTable.bzseq[producerId]. 

While the seqTable.bzseq[producerId] value Is reset, each node's seqTable.busyQ and 
seqTable^tO fields must be preserved. They represent the state necessary to accept 
or reject packets in the CRC error retry loop with die correct ack done or busy 
re^onse for this destination. In this example, node B rejects sendA^esponse packet 22, 
while node C accepts as new both packets 21 and 23. 

The sender must resend all packets flmt had been transmitted up to the point that it halted 
new packets, in this case through packet 23. "With no erroors, nodes B and C track the 
sequence cumbers. 

Case 2: CRC error on acknowledge packet from node C, detected at node A. 
Packet sequencing from standpcunt of the send packet: 



Packet sequencing from standpoint of the send paekg^; 
Node A send vttluftg! 



Busy retry 



B 1 


C 1 


B 


1 


B I 


C 1 


B 


1 


BIB 


21 


22 


23 




21 


22 


23 


* 


24 2S 








• 








• 




0 


0 


0 


m 


0 


0 


0 


* 


1 1 
















* 




21 








21 . 






• 


24 










A 










^_ 


~ — 


-+1 


































Node A recftive values! U enorloop— >l 

^ k-btyloop— >l 

CRC error detect _22 cir^ a 

A 

BusydDiBBt: ..21 bsy-*^-23bzy— — ~— ~^ 
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Node B valBts; 

Packet ocmplelion; 





Bry • 


Bry 




Bry * 


Ddce 


Dooe 


Acocp1/ic)ect: Rej 


Rej • 


Rej 




Rej ♦ 


Acpc 


AcpL 


scqTsblcscqCprodiiccrld]: 














20 21 


22 ♦ 


23 


21 


22 * 


23 


24 


seqTablcbzscq^oducciId]: 














O A 




u 


u 


0 * 


0 


1 


seqTablewlwsybzrcxhioBrld]: 














0 1 


1 • 


1 


1 


1 * 


I 


0 


scqTable.tq)C[|inx}uoerId}: 














20 21 


21 ♦ 


21 


21 


21 ♦ 


21 


24 


Node C values: 




























(CRQ 






Done 








AooBpt/mjKiz 






AgiA 








seqTable^Eprodvcerld): 














20 21 


21 • 


21 


21 


22 - 


23 


24 


$e4Table.l>z9eq[pnxhiccrU]: 














0 0 


0 * 


0 


0 


0 * 


0 


1 


»qTflble.bi8y(prodi]oerId]: 












9eqTflfcle.aG])t[pnDduoerid]: 














20 21 


21 • 


21 


21 


22 ♦ 


23 


24 



This example illustrates the inability of the sender node to detect whether a packet has 
been accepted at its destination. Since node C in this example has seen no error, both packets 21 
and 23 have valid pLabeLseq fields and am given ACK_done responses. 



Case 3: CRC error on send packet to node C» detected at node C before begmning 
busy retiy loop for node B. 

Packet seouencing from standpoin t of the send packet- 

Desdaatian: BICIBIBICIBIBIB 
pLabel^: 21 22 23 * 21 22 23 * 24 25 

pLabel.bzseq 0 0. 0*0 0 0*1 l 
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GRCeirictzy: 21 
I 



21 

A 



Busy leoy 



Nfiflfr A my.ivfi yaIum: 

CRC ttrcr detect 
detect: 

Node B values; 

PAckst ccmpletloa: 
Bzy 

AcccptABject Ri^ \ 

seqTia>le.seq(]»t)duoeTld]: 

20 21 
s6qXVtble.bzseq[jpatxluccrId]: 

0 0 
scqTablcbosyCproducerld]: 

0 1 
6eqTablA.aq)t(pno(hiGBrId]: 

20 21 

Ncxie C valuftit; 
Fickec coapletaoo: 



|< 

A 

...22 en^ 
„2i bzy— ^-TShiy- 



AcoeptAeject: 



(CRQ 



seqTabk,jeq[pradticexZdj: 

20 21 

aeq7)ible.tnseq|pKKluoerId]: 
0 0 

seqTabie.biuy|prodiBerltf]: 

se(^IUs]e.tcptIjpRidaoer1d(): 
20 21 



Rej 



22 

0 

1 

21 



21 
0 

21 



Bzy 



23 

0 

1 

21 



21 

21 



21 
0 
1 

21 



Dooe 
Acpi 

21 
D 



21 



24 

A 



k-teyJoop— >l 

A 



Bay 



22 

O 

1 

21 



22 
0 

22 



Dooa 
Acpt 



23 

0 

1 

21 



23 
0 

23 



Dooe 
Acpi 



U 
1 
0 
24 



24 

I 

24 



This example is based on the fbUowing sequence of events: 

Node B is sent 2 sendAesponse packets, 21 and 23, which it rejects &s busy. 
Node C is sent a packet 22 which it never sees because of a CRC error at its node; no eiror 
is detected at node B. 

Node A b^ins its busy retry for node B packets by first waiting for the completioD of out- 
standing Acks. Now node A detects the CRC error. It begms a CRC error loop, whfch 
must include all packets trcm 2 1 (last good packet before the enrw) through 23 (last 
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packet sent before the eiror is detected). 

In this example, whea the CRC error retry loop is entered, retried packets to node B in the 
error loop are rejected based on the pLabd.seq value and node B's seqTahle.busyO and 
£eqTable.acptD fields. Since the seqTable.busyQ bit is set, packets are rejected; since 
seqTable.busyD is set but both packets had originally been marked "busy" (based on comparison 
of tfie pLabeLseq value with the seqTable.acptn field), both retried packets 21 and 23 are 
acknowledged as "busy**. 

The seqTable.bzseqlproducerld] field from node A makes a tiansition when the busy loop 
is finally started and packet 21 is retransmitted as packet 24. Node B detects this transaction and 
now evaluates whether packet 24 can be acoqpted, based on whether its recewe buffer is fiiU. In 
the assumed example, both retried packets 24 and 25 are accepted. 

The basic concept for initiating a busy retry loop in the potential presence of errors is to 
wait in starting the loop until acknowledgments for all packets prior to the fi«t retried busy packet 
are received and verified as good. This is necessary since changing the state of the pLabel.bzseq 
bit causes aU node packets to reset tiheir seqTable.busyD and seqTahle.acptD fields. These fields 
had maintained enror retry state for generating packet accepl/^jcct and acknowledgments prior to 
the busy retry loop. These fields are reset only when node A verifies that no ciror conditions 
remain 
outstanding. 

Producer nodes: Request Queue State 

In a prefefted enibodiment,, each producer node may have up to the following three types 
of outgoing queues in addition to a register, and two types of incoming queues or strucnires (see 
Figure 39) 
Outgoing: 

1. a new request queue of unissued requests, presumably in the transaction layer (above 
the link layer); 

2. a reqponse-pending queue of unissued response packets to other nodes, again in the 
transaction layer; 

3. a "sendjpkt'* queue of send and response packets in process, which are issued to the 
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ringlet, in the link layer; 

4. a "rcnryjjkt- queue of busy packets to retry [N.B. this applies only to a busy retry 
embodiment]; 

5. a * retry_pkt- register, to both fill the retry j>kt queue and to recycle packets for error 
retry to the sead^pkt queue. 

Incoming: 

1. a queue of incoming acknowlcdgxnenU to this node in the link layer; 

2. a structure for requests which are awaiting responses, in the transaction layer. 

For outgoing send and respcmse packets in the transaction layer, the queues may be in 
some cases niultq>le queues, where there is no ordering dependency between the queues. An 
example is the multiple ports of a switch. 

For response packets to this node in SSO or<tering mode, responses from a given destina- 
tion node will always ccanc in order, while no ordering is guaranteed for responses between dif- 
fcrent nodes. Consequently, the send packet structure awaiting responses is shown as a structure 
rather than a queue. 

Rest>onse packet uMffti in SSQ wrieriny spaces 

IEEE P1394.2 spedlies two types of operations to write data to a destination node: a 
re^nselcss move transaction and a write transaction with a response. The SSO ordering struc- 
ture guarantees (xdered deUvety of data; what then is the difference between move and write 
transactions? 

Write send packets provide a positive indication, generated by the destination node, that 
die write data actually reached this node. That is. it positively indicates completion of the opera- 
tion. Move send packets have no positive indication of ccmplction. 

There ere two questions that arise from this disdnction: 

1. What can the positive indication of can^>letion be used for? 

2. What architectuiBl extensions can be combined with move transacdons to ensure that 
ordering is maintained in an imperfect link topology, specifically covering invalid 
addresses and "stuck at" receiver nodes that refuse to accept new packets? The posi- 
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tive indication of completion for the write truuftctioii wiU be visible to the tmnsacti on 
layer. Whether it can be visible at some system level above this depends on a transac- 
tion level architecture ttiat is beyond the scope of this discussion. The point of SSO 
ordering is that no structure is necessary to ensure ordering. 
Here ore two possible uses for write transactions within the transaction layer; this list is 
hardly exhaustive. First, write transacdons can be used as the basis for a reliability architecture. If 
a send packet failed to receive a reqxmse in a specified period of time, a node timeout could be 
reported. In this usage, there is no a priori requirement for response completion prior to starting 
the next send packet. 

In the second use for write transactions is to use them as a basis for high availability archi- 
tecnires. In this application, the response to the write signals the cosnpletion of a "write oonimit" 
to stable storage. The problem with this is what to do with tlie response. Does one use it to gate 
transmitting the next send packet? If so, this dramatically slows down the packet delivery rate. 
Does the system architecture support a mechanism to use the response as a completion barrier 
indicator^ Most Gnduding UFA) do not. 

Send packet qwuc ctnictttTfi 

Since the addz«ss spaees to PI394:2 may, in general, be both otdered and non-ordered, the 
node must maintain a structure of data for each command to identify and support both addressing 
models. This proposed structure consists of three fields: 



Send Dkt SSO st:rnrt-iir«> 


■ one per send or response packet 


sendj>kt . val 


(1) 


/♦ 


Entry Valid ♦/ 


send^pkt.sso 


(1) 


/* 


From command address space */ 




(1) 


/• 


ACK_don© recvd^ response may be reqd 


send^kt - busy 


(1) 


/♦ 


ACKJbiisy received, awaiting retry ♦/ 


sendjpkt . tnit_er 


(1) 


/* 


Initial (first) packet in error retry loop •/ 


send^pkt . no_xiait 


(1) 


/* 


No transmit to ringlet during error loop */ 


send_pkt . err_ret 


(1) 


/♦ 


Error retry state in error retry loop ♦/ 


3end_pkt . seq 


(6) 


/• 


Seq number for this send/response pJct ♦/ 


send_plct .packet 


(N) 


/*N bit aend / receive packet */ 
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Other control bits may be used for impkmentation purposes beyond SSO ordering as described 
here. 

Conceptually, sendjjktO may be viewed as a FIFO queue, with newly sent packets enter- 
ing at the top and concluded packets coming out the bottom as acknowledgment padoeU in a car- 
responding HFO arc received. Any send packets acknowledged as "ack^done" may in turn may 
be awaiting a response. 

In addition to the scnd_pkt FIFO queue, other queues, registers, muxes, state machines, 
and control logic are described in the supporting flow diagrams. 

Producer nodft pointers for CRC «rnr l#v>p 

Certaun producer node variables arc needed for C3^C error loop and busy retry loop han- 
dling. Global variables for CRC enror loop operadon include these, where the prefix "or_" indi- 
cated "micrcd". 

cr^la3t_ct (fi> /• Sequence count for latest SSO ordered pkt 

sent to the local ringlet */ 
no_laat_ct (6) /* Sequence count for latest non-SSO ordered 

pkt sent to the local ringlet ♦/ 
or_rcv<l_Ct (6) /* Sequence count for latest SSO ordered pkt 

received from the local ringlet */ 
When a packet is seat to the local ringlet. itspLabel field is set to correspond with 
send_pkt fields, which arc carried in the packet path into the sencLpkt register. In general: 
pLabel.sso - sendj>kt_reg.sso 

pLabel.bzseq ^ bzseq /* Global producer node state bit */ 

pLabel.seq send_pkt_reg,3eq 
What value is assigned to pLabel.seq depends on the operation (e.g,, new packet, hyxsy retry 
packet, or CRC retry); see details below. 

Recdver pointer conmarison: supporting a wra pped modulo 64 coun^ 

Prom the "Pjroposal overview" section above, it was noted that three comparisons can be 
iniportant to the receiver operation: 

1. The receiver node can detect a valid new, SSO ordered packet from a^ven produccrld 
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if: 

pLabel.seq •» seqTable.3eq[producerid] + 1, and 
pLabel.seq > seqT able . acpt [producer Id] 

2. The seqTabIc.seq[producerId] field is reset to the current pLabcl.seq value when this 
node detects that this producerld is executing a CRC error loop. This is indicated by 
cGinparing the seq fields for a packet - or acknowledge - from this node: 

if {pLabel.seq <- seqTable . seq [producerld] ) then 
{seqTable. seq [producer Idl = pLabel.seq 
) 

3. Finally, each node can detect a packet with a sequence eiror firom the producerld if: 

pLabel.seq > seqTable.seq[produccrId] ^ I 

Supporting both ''greater than" and "less than or equal" comparisons with a modulo 64 
counter can be achieved if the producerld ensures that the comparison will never exceed a differ- 
ence at the receiver node of 31 and the count is defined as below: 

"A<=B"if: 

- high Older Ut equal and A <:s B with lower order tuts 

- high order bit unequal and A > B with remaining bits 

"A > B" if: 

- high OTder bit equal and A > B with lower order bits 

• high order bit unequal and A B with remaining bits 

This section provides additional explanation and emphasis on some subtle details with 
possible implemeotationa of the invention. 

The first of these problems dealt with reliably resetting appropriate seqTable[producer]d] 
fields following a CRC error. Once a CRC error has occurred, the fieUs in the seqTable.seq[p(ro- 
duccrlD] for every node in the ringlet may contain values which are not in synch with otfier nodes 
within the ringlet 

At the same time, since the seqTableD values are ♦♦only** updated when a valid, cor- 
reedy sequenced packet is identified at the receiver node, then the vahies must be considered as 
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coiTcct insofar as this node is concernecL They may. however, be out of synch with respect to both 
the sending node and other ringlet nodes. 

So the task of the produccrld node is to reliably get all nodes back into synch. The prob- 
3an is in achieving this reset: what if the first packet in Ac error loop itself has a CRC error? 

A simple solution to this problem is for the produccrld node perf onning the error handling 
loop to begin its enror loop by transmitting - in non-pipelined mode - the last known good, com- 
pleted packet. The produccrlD node would retry this packet until it received a valid acknowledg- 
ment. Once this is received, then every node on Ae ring must have its seqTable.seqD field for this 
producerlD set to the same value. The same argument holds for rescttuig the seq'Rible.bzseqn 
field. But the scqTable busyQ field reflects the sUte of the acknowledgment to the last valid, com- 
pleted packet deUvered from this produccrld to this receiving node. In considering how to handle 
the next vaUd packet (any retried packet wiU be rejected), this field has validity. This field, conse- 
quently, must not be reset in the reset interval. Similarly, the seqTable.aqjtQ field is also left 
unmodified. 

Note that by retrying a known good, completed packet (which wouki be acknowledged as 
valid and rejected by the target node in the ringlet) the produccrlD node is tnmsmittxng a packet 
for only its the packet sequence number, rather than its data content. 

The same approach of rcsending a known packet undl a valid ACK is returned, can be 
used for ringlet initialization to initialize the above mentioned fields. But other fields, such as 
seq-ftblcbusyn and scqTablcacptD, must be reset by other means, such as using a non-SSO 
space to write into the seqTable.acptO field the value 6>Label.seq). 

The second problem with the original iroposal dealt with the questicn of ambiguity in the 
producerlD node determining whether packets in its error loop had been completed or noL That 
is, if any (sendAeceivc) and acknowledge pair is found to have a CRC error, then the producerlD 
node cannot determine whether the CRC enror occurred before the packet reached its ringlet desti- 
nation, or after. So it cannot reliably mark a retried packet as "completed". 

The solution to this problem was to add the (6 bit) seqTable.acptQ field to each receiver 
node s seqikble array to record the last valid sequence number detected at this node prior to hav- 
ing its seqTablcscqD reset by a packet with a lower value pLabeLseq from this produccrld, or the 
sequence number of the first packet rejected for busy rehy (indicated by the seqTable.busyO field. 
Note that tracking the seqTable.acpt[] field means that 
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1 , The last accepted packet at this node must have a sequence number <^ seqTablcacptQ. 
so any new. unseen packet must have a sequence value > scqTablcacptO, provided that 
seqTable.busyD is reset 

2. The scqTkblc.acpt, in the absence of errors or busy packets, tracks the current pLa- 
bcLscq value. This fact is cnjcial in creating a modulo 64 rolling count definition. 

Notes regarding the Example of Fipirea 

Rguies 39-60 show the state of the system of the invention at successive stages of process- 
ing packets wherein a busy condition has arisen with respect to a consumer node* a CRC (or 
other) error condition has also arisen, and the "busy- node is not weil-behaved - i.e. it has been 
persistently busy for longer than some predetemnined length of time or number of packets sent or 
received, or it is erroneously transmitting ack_busy responses. 

In die example of Figures 39-60, it will be noted that the or Jast_ct is the count number of 
the last ordered packet sent, and that or_rcvd_ct refers to the count of the last ordered packet 
received. The or_n5vd_ct value is frozen if a busy retry is required (which is detected in the 
rcsend logic). 



For this example, it is assumed that there are previous busy packets, e.g. 54 and 59. which 
are in the rctry_pkt queue. Note that 

(or_la5t_ct - or_rcvd_ct) = (23-59) modulo 64 
= 28 

and 28 is less than or equal to the threshold for the difference (orJlast_ct - or_rcvd_ct). 
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What is claimed is: 

I . A system for maintaining a sequence of packets transmitted by at least one producer node 
to at least one consumer node in a computer network supporting , including: 

a first send subsystem of said producer node configured to maintain sequence and packet 
state inforaiadon for at least one said packet sent by said producer node; and 

a first receive subsystem of said producer node configured to maintain sequence and 
packet sute information for an acknowledgment sent by said consumer node to said producer 
node, and to detect a condition wherein a sent packet has not resulted in an acknowledgment 

a second receive subsystem configured of said producer node configured to detect busy 
acknowledgments irom said consumer node; 

a second send subsystem of said consumer node configured to maintain sequence and 
packet sUte information for said acknowledgment; 

a second receive subsystem of said consumer node configured to maintain sequence and 
packet state informadon for said packet sent by said producer node, and to maintain overall 
sequence state information and packet accept state infonmation f<x all packets received by said 
consumer node from said producer node, and further configured to reject packets when at least 
one said busy acknowledgment is detected; and 

a node monitoring subsystem in said producer node, configured to determine when at least 
one said consumer node in said network fails to xiiect a piedetemiined msponse criterion. 

2. The system of claim 1. wherein said node monitoring subsystem includes a busy acknowl- 
edapient queue configured to store a plurality of busy acknowledgments received fr<wn said con- 
sumer node. 

3. The system of daim 2. wherein said node monitoring subsystem is configured to make 
said determination based upon said busy acknowledgment queue storing at least a prcdetennined 
xuimber of said busy acknowledgments. 

4. The system of claim 1 , wherein said predetermined response criterion includes a predeter- 
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mined amount of time during which said producer node receives a plurality of busy acknowlcdg-* 
ments from said consumer node. 

5. The system of claim 1, wherein said predeterauned response criterion includes a predeter- 
mined amount of time during which said producer node receives no acluiowledgmem from said 
consumer node. 

6. The system of claim 1 , wherein said system is configured to process at least a first packet 
comprising a nonidempotent request while maintaining unaltered a state of a node to which said 
firrt p n ciffgt is midr essed, upon receiving said first packet more dian once. 

7. The system of claim h including a retry subsystem configured to retiy at least one packet 
including a nonidempotent request wtule maintaining onaltered a state of said consumer node 
upon reeeiving a second instance of sud retried packet 

JS. The system of daim 1. wherein said network is a ringlet n a twuik > 

9. The system of claim 1, wherein said network includes at least one ordered node configured 
to maintain strong sequential ordering and at least one unordered node configuied in a manner 
other than to maintain strong sequential ordering. 

10. The system of claim 1, wherein said network includes at least a first node configured to 
support nonidempotent commands and second node configured not to support nonidempoient 
commands. 

11. The system of claim 1, wherein said network includes at least one dynamic node config- 
urable, at different times, both in a manner to maintain both strong sequential ordering and in a 
manncLDther than to nudntain strong sequential ordering. 

12. The system of claim 1, further including: 
an error detection subsystem; and 
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a reset subsystem configured^ upon detection of an error by said tcrw detection subsystem 
upon sending of a current packet, to reset said sequence of each said ccMisumer node to a common 
network value. 

13. The system of claim 12, wherein said reset subsystem is configured to cause said producer 
node to resend a previously sent packet at least once, and tepeatedly as required, until a valid 
acknowledge packet is received by said producer node. 

14. The system of claim 13, wherein said previously sent packet is a packet for which said 
producer node has received an acknowledge done packet. 

15. The system of claim 14, further inchiding a first retry subsystem configured to send retry 
packets between said previously sent packet and said current packet, inclusive. 

16. The system of claim 15. further including a retry packet validity check subsyBtem config- 
ured to determine validity of retry packet acknowledgments received at said producer node in 
response to sending said retry packets. 

1 7 . The system of claim 16» further including a second retry subsystem configured to send 
said retry packets when said retry packet validity check subsystem deteimines that aretry packet 
acknowledlgment has an invalid status. 

18. The system of claim 13, wherein said reset subsystem is configured to remove a data field 
firom said previously sent packet before resendiqg. 

1 9. The system of claim 1 3. wherein said reset subsystem includes: 

a sequence validity state subsystem configured to maintain a sequence validly state value 
at each said coosumcr node, and to reset each said sequence validity state value to a common 
value upon receipt at each said consumer node of said previously scat packet 

20. Th^ system of claim 15, wherein said retry subsystem includes: 
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an acceptance validity state subsystem configured to maintain an acceptance validity state 
value at each said consumer node; 

an acceptance validity compaiison subsystem configured tc generate a comparison of an 
acc^tance validity field of said retry packets with said acceptance validity state value at each said 
consumer node; and 

a reny packet reject subsystem configured to reject each said retry packet whose said com- 
parison meets a predeteraoined criterion. 

21. The system of claim 1. including a plurality of send and consumer nodes, wherein each 
said consumer node includes one said receive subsystem configured to read state infomiation of a 
plurality of said packets and said acknowledgments, respectivdy, even when saki packets and 
acknowledgments are addressed to nodes other than the consumer node reading said state infor- 
madon, to maintain common sequencing informadon among all nodes that support strong 
sequence ordering. 

22. The system of claim 1, including a plurality of sa^d producer nodes and a plurality of said 
consumer nodes* wherein: 

at least a subset of said producer and ccmsumer nodes are configured as su^ng sequential 
ordering (SSO) nodes to send and receive, respectively, SSO packets^ and to receive and send, 
respectively* SSO acknowledgments of said SSO packets; and 

each said producer and consumer node of said subset is ecmfigured to read said SSO pack- 
ets and/or said SSO acknowledgments. 

23. The system of claim 1, configured to maintain unaltered an order of request packets as 
transmitted by producer nodes on said network, and to maintain unaltered an order of response 
packets generated in response to said request packets by consumer nodes on said netwoik. 

24. A system for maintaining a sequence of packets transmitted by at least one producer node 
to at least one consumer node in a computer network, including: 

a sequence check subsystem in said consumer node configured to determine whether at 
least one packet in the sequence is in a valid position in the sequence; and 
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a busy acknawlcdgmcru detcctioD subsystem in said producer node configured to deter- 
mine whether a received acknowledgment indicates a busy condition in said consumer node; and 

a node maniioring subsystem in said pnxiuccr node, configured to detcnninc when at least 
one said consumer node in said network fails to meet a predetennined response criterion. 

^5. Tie system of claim 24, wherein said node monitoing subsystem includes a busy 
adcnowled^jncnt queue configured to store a plurality of busy Bcknowlcdgmcnts received from 
said consumer node;. 

26. The system of claim 25, wherein said node monitoring subsystem is configured to make 
said determination based upon said busy acknowledgment queue storii^ at least a predetermined 
number of saidSusy acknowledgments. 

27. The system of claim 24, wherein said predeteimined response criterion ^y^hidgg a picde- 
tenmined amount of time during which said producer node receives a piurali^ of busy acknowl- 
edgments from said consumer node. 

28. The system of claim 24, wherein said prcdcterinincd response criterion includes a prede- 
termined amount dmc during which said producer node receives no acknowledgment from said 
consumer node, 

29. A system for nmintaiining a sequence of packets transmitted by at least one producer node 
to at least one consumer node in a computer network, including: 

a pipelining subsystem configured to pipeline packets from said producer node in the 
absence of acknowledgment to said producer node of previously transmitted, unprocessed pack- 
ets: 

a counter subsystem configured to determine a sequence count difference between a first 
said unprocessed packet and a current packet; 

a cutofip subsystem configured to terminate said pipelining of said packets when said 
sequence count difference reaches a predetermined threshold; 

a busy detection subsystem configured to detect a busy condition at sakl consumer node; 
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a packet suspension subsystmn configured to suspend pipelining of said packets when said 
busy condition is detected; and 

a node monitoring subsystem in said producer node, ccmfigured to determine when at least 
one said consumer node in said network fails to meet a predetemined response criterion. 

39, The system of claim 29, wherein said node monitoring subsystem includes a busy 
acknowledgment queue configured to store a plurality of busy acknowledgments received from 
said consumer node. 

31 . The system of claim 30, wherein said node monitoring subsystem is configured to make 
sakl determination based upon said busy acknowledgment queue storing at least a predetermined 
number of said busy acknowledgments. 

32. The system of claim 29, wherein said predctcmiined response criterion includes a prede- 
tcimined amount of time during which said producer node receives a plurality of busy acknowl- 
edgments itom said consumer node, 

33. The system of claim 29, wherein said predetermined response criterion includes aprede* 
termined amount of time during which said producer node receives no acknowledgment from said 

^ consumer node. 
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Abstract of the DiacicMiirr 
A system for mainteining reliable packet distribution in a ring network with support for 
strongly ordered, nonidexnpotent commands. Bach consumer node on the network maintains a 
record of the sequence of packets that have passed through that node, and die state of each of the 
packets at the time it passed through, including a record of the last known good packet and its 
sequence number. When a producer node detects an error condition in an acknowledgment for a 
packet, resends all packets beginning with the Ust known good packet Each consumer node is 
able to process or reject the reseat packets, including packets that may already have been pro- 
cessed, which it is aware of due to the packet and state records for all packets. Strong ordering is 
thus supported, since the sending order of packets can be maintained for processing due to the 
resends, and non]dcnq>otent cnnmands aje supported due to the consumer nodes* ability to deter- 
mine whether they have already processed a pyen packet, and to send an acknowledge^one reply 
if that is the case. Hie system is equipped to operate sucoessfiiUy in the presence of a failed or 
very busy node by maintaining a queue of busy acks from each node, or from nodes thai are par- 
ticularly busy, and when the queue is full taking steps to bypass the busy or failed node. Alterna- 
tively, the system can detect that same prcdetcmiined period of time has passed during which 
either no response or only busy responses have been received from a given node, and at that point 
determine to effectively bypass the node for future transactions, at least for acme timeout period, 
so that packet transactions for the rest of die network can proceed unUndeied by the ill^-behaved 
node. 
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